-- Logs begin at Fri 2022-06-17 10:52:21 UTC, end at Fri 2022-06-17 12:05:55 UTC. -- Jun 17 11:24:06.547151 dv-cinder-837356-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 17 11:24:06.554590 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 17 11:24:06.557220 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 17 11:24:06 2022] *** Jun 17 11:24:06.557349 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 17 11:24:06.557455 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 17 11:24:06.557559 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: nodename: dv-cinder-837356-3-iscsi Jun 17 11:24:06.557660 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: machine: x86_64 Jun 17 11:24:06.557769 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: clock source: unix Jun 17 11:24:06.557893 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: pcre jit disabled Jun 17 11:24:06.558003 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: detected number of CPU cores: 4 Jun 17 11:24:06.558104 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: current working directory: / Jun 17 11:24:06.558204 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: detected binary path: /usr/bin/uwsgi-core Jun 17 11:24:06.558309 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: your processes number limit is 47680 Jun 17 11:24:06.558404 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: your memory page size is 4096 bytes Jun 17 11:24:06.558510 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: detected max file descriptor number: 2048 Jun 17 11:24:06.558613 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: lock engine: pthread robust mutexes Jun 17 11:24:06.558866 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: thunder lock: enabled Jun 17 11:24:06.559167 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 17 11:24:06.559346 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 17 11:24:06.603610 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: Python main interpreter initialized at 0x55d1b84fa9c0 Jun 17 11:24:06.603610 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: python threads support enabled Jun 17 11:24:06.603610 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: your server socket listen backlog is limited to 100 connections Jun 17 11:24:06.603610 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: your mercy for graceful operations on workers is 90 seconds Jun 17 11:24:06.603994 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: mapped 403077 bytes (393 KB) for 2 cores Jun 17 11:24:06.604075 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: *** Operational MODE: preforking *** Jun 17 11:24:06.604293 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: *** uWSGI is running in multiple interpreter mode *** Jun 17 11:24:06.604293 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: spawned uWSGI master process (pid: 80090) Jun 17 11:24:06.604502 dv-cinder-837356-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 17 11:24:06.605598 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: spawned uWSGI worker 1 (pid: 80102, cores: 1) Jun 17 11:24:06.605598 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: spawned uWSGI worker 2 (pid: 80103, cores: 1) Jun 17 11:24:06.605598 dv-cinder-837356-3-iscsi devstack@keystone.service[80090]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 17 11:24:12.094614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:24:12.094614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: from cryptography.utils import int_from_bytes Jun 17 11:24:12.095393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:24:12.095393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: from cryptography.utils import int_from_bytes Jun 17 11:24:12.095508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:24:12.095508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: from cryptography.utils import int_from_bytes Jun 17 11:24:12.095913 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 17 11:24:12.095913 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: from cryptography.utils import int_from_bytes Jun 17 11:24:12.148475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 11:24:12.148676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 11:24:12.148844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] command line args: [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 11:24:12.149111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 11:24:12.149294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ================================================================================ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 11:24:12.149550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] admin_token = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.149795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_dir = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.150012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.150194 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_source = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.150375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] debug = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.150574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.150946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.151126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.151377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.151625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.151856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.151941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 17 11:24:12.152065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_config_append = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.152149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 17 11:24:12.152218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.152280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] command line args: [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 17 11:24:12.152406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_dir = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.152494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 17 11:24:12.152557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.152680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ================================================================================ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 17 11:24:12.152804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.152919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] admin_token = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config_dir = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config_source = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] debug = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.153987 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.154968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.155089 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.155210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_config_append = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.155331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.155445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.155445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_dir = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.155672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.155851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.156102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.156908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.157173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.157354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.157533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.157782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.158061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.158299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.158536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.158772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.159006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.159257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.241653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.241969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.242282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.242825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.243023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] publish_errors = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.243276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.243527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.243718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.243980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.244227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.244473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] standard_threads = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.244715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.244969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.245097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.245179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.245320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_journal = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.245440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.245534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_json = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.245666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_stderr = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.245870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_syslog = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.246006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.246086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.246237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] publish_errors = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.246358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.246438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.246605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.246685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.246865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.246944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.247108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.247193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.247366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.247444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.247609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.247688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] standard_threads = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.247852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.247929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.248094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.248178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.248262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.248334 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.248442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_journal = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.248589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.248676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_json = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.248789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_stderr = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.248930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.249048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_syslog = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.249169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.249291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 17 11:24:12.249407 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.249521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.249646 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.249768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.249901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.250886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251715 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.251908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.252999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.253973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.254814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.255961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.256915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.257946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.258885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.259955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.260904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.261024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.261124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.261234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.261346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.261454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.261558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.261684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.262922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.263970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.264876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.265001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.265118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.265233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.265347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.265463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.265586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.266216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.266332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.266460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.266577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.266693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.266820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.266943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.267897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.268958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.269082 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.269199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.269320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.269437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.269552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.269744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.269911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.270887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.271951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.external = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.272910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.password = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.token = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.273981 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.274986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.external = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.275934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.password = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.token = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.276900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.277996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.278228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.278344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.278460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.278582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.278699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.278827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.278945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.279908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.280956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.281073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.281189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.281377 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282008 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.282866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.283928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.284825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.285898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.286957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.287914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.288037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.288160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.288277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.288394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.288511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.288885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.289998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.290955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.291911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.292976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293334 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.293946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.294889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.295945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296194 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.296921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.297888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.298947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299396 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.299870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.300954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.301962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.302992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.303939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.304991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305701 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.305963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306082 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.306901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.307911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.308972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.309966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.310923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.311893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.312009 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.312124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.312240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.312364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.312972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.313920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.314986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.315923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.316980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.317977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.318920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.319978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.320906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321858 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.321976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.322940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.323927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 17 11:24:12.324980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 17 11:24:12.351283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.351454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.351674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.353582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.353816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.354066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.359956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.362850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.387812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.388047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.388870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.389106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.441321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.442156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.489973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.490203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.499835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.500766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.558371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.558596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.558821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.559044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.559255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.559492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.560914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.561146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.561370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.561588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.561804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.562045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.583597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.583755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.583843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.583984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.584921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.585051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.595879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.596038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.608536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.608645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.608779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.608943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.609901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 17 11:24:12.627434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.627645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:24:12.627733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.627935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:24:12.628061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.628185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:24:12.628373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.628477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.628578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 17 11:24:12.628680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.628791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.628893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.629006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.629155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.629266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.629420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.640021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.640207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.640298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.640482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.640567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.641364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.641454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.641644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.649618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.649817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.652906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.653085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.653169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.653363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.653447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.653645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.653729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.653946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.654033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.654225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.654322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.654479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.654565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.654743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.654827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.655010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.655102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.655273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.655355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.655537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.655618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.655797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.682830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.682929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.684461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.684553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.684740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.684826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.690608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.690704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.690999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.691086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.691278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.691363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.698495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.698587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.698773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.698870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.703236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.703329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.703518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.703603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.703797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.703882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.704969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.705145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.705236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.723538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.723694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.723816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.723977 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.728280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.728483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.731880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.732107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.732275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.732506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.737686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.737986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.741335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.741610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.745015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.745312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.745393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.745595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.745680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.745896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.745997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.746176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.746261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.746453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.746536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.746736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.767275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.767552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.767678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.767819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.767962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.768091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.768233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.768497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.775914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.776474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.777604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.777920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.778201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.778493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.781048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.781351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.781631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.781721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.781944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.782044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.782222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.782307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.782512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.782789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.795650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.795929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.796208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.796299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.796485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.796577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.797411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.797508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.797630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.797716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.797924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.798024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.798518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.798804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.817001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.817159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.820443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.820581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.823732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.823885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.825494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.825594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.827158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.827247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.827438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.827524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.827721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.827817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.834974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.835068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.836554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.836683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.839800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.839890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.840094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.840179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.840485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.840582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.840763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.840846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.849826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.850078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.850164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.850364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.850450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.850646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.858172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.858450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.858589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 17 11:24:12.858728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.858812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.859001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.859084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.859271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.859873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.860081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.860165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.860436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.860526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.860713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.860798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.860990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.861084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.861262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.861347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.861529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.861610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.861802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.861927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.862089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.862173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.862347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.862430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.862604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 17 11:24:12.900011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d1b84fa9c0 pid: 80102 (default app) Jun 17 11:24:12.900569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d1b84fa9c0 pid: 80103 (default app) Jun 17 11:24:12.903442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203644f-6a3a-46e7-92ee-243969da588b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:12.903739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203644f-6a3a-46e7-92ee-243969da588b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:12.904015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203644f-6a3a-46e7-92ee-243969da588b None None] PATH_INFO: `/v3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:12.908229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Fri Jun 17 11:24:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:20.667506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0bd1a-fcbe-4005-b35f-484f35ed503e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:20.668013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0bd1a-fcbe-4005-b35f-484f35ed503e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:20.668115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0bd1a-fcbe-4005-b35f-484f35ed503e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:20.672461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:24:20.677916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:20.678306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:20.678488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:20.747827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: INFO dbcounter [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] Registered counter for database keystone Jun 17 11:24:20.769726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG oslo_db.sqlalchemy.engines [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80102) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 11:24:20.775947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writer thread running {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 11:24:20.956700 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG passlib.handlers.bcrypt [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80102) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 11:24:20.957048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG passlib.handlers.bcrypt [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80102) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 11:24:21.012155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:21.013774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:21.142883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 17 11:24:21.321950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0bd1a-fcbe-4005-b35f-484f35ed503e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:21.393215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: INFO dbcounter [None req-80f0bd1a-fcbe-4005-b35f-484f35ed503e None None] Registered counter for database keystone Jun 17 11:24:21.412284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG oslo_db.sqlalchemy.engines [None req-80f0bd1a-fcbe-4005-b35f-484f35ed503e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80103) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 17 11:24:21.419147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writer thread running {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 17 11:24:21.539047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0bd1a-fcbe-4005-b35f-484f35ed503e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:21.543359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfb0e5-2b05-4713-b2f6-a30d1070bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:21.544022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfb0e5-2b05-4713-b2f6-a30d1070bac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:21.544312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfb0e5-2b05-4713-b2f6-a30d1070bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:21.544590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfb0e5-2b05-4713-b2f6-a30d1070bac9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:21.552746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfb0e5-2b05-4713-b2f6-a30d1070bac9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:21.604511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fcbfb0e5-2b05-4713-b2f6-a30d1070bac9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 11:24:21.604806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfb0e5-2b05-4713-b2f6-a30d1070bac9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:21.614964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Fri Jun 17 11:24:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:24:21.620863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:21.635299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a0639-3c93-4adb-a3b6-cac33daccd22 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:21.639711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71062005-c203-4c0b-ad09-a114e733244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:21.640816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71062005-c203-4c0b-ad09-a114e733244a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:21.641104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71062005-c203-4c0b-ad09-a114e733244a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:21.641381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71062005-c203-4c0b-ad09-a114e733244a admin admin] PATH_INFO: `/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:21.642924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71062005-c203-4c0b-ad09-a114e733244a admin admin] RBAC: Authorizing `identity:get_project(project_id=5460a823bf554ac5a99fd5d73c21c6d1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:21.693583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING keystone.common.rbac_enforcer.enforcer [None req-71062005-c203-4c0b-ad09-a114e733244a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 17 11:24:21.694133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71062005-c203-4c0b-ad09-a114e733244a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:21.698598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Fri Jun 17 11:24:21 2022] GET /identity/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:23.775708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5b99a-aff7-41ad-ab2f-8942eabd2326 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:23.776279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5b99a-aff7-41ad-ab2f-8942eabd2326 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:23.776569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5b99a-aff7-41ad-ab2f-8942eabd2326 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:23.780755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:24:23.786330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e4f4c-03af-41b8-a76c-0b5be987d5c5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:23.786721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e4f4c-03af-41b8-a76c-0b5be987d5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:23.786904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e4f4c-03af-41b8-a76c-0b5be987d5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:23.817190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d05e4f4c-03af-41b8-a76c-0b5be987d5c5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:23.818772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d05e4f4c-03af-41b8-a76c-0b5be987d5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:23.826717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 17 11:24:23.850181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5b99a-aff7-41ad-ab2f-8942eabd2326 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:23.862117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5b99a-aff7-41ad-ab2f-8942eabd2326 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:23.866438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c5030-1e08-480e-9a3b-a08c323b64f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:23.867071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5030-1e08-480e-9a3b-a08c323b64f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:23.867355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5030-1e08-480e-9a3b-a08c323b64f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:23.867631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5030-1e08-480e-9a3b-a08c323b64f8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:23.875719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c5030-1e08-480e-9a3b-a08c323b64f8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:23.877471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c5030-1e08-480e-9a3b-a08c323b64f8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:23.886116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Fri Jun 17 11:24:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:24:23.891955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e4f4c-03af-41b8-a76c-0b5be987d5c5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:23.900997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e4f4c-03af-41b8-a76c-0b5be987d5c5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:23.905399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2691df-54d5-4234-bf06-53098d59224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:23.906303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2691df-54d5-4234-bf06-53098d59224f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:23.906592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2691df-54d5-4234-bf06-53098d59224f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:23.906869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2691df-54d5-4234-bf06-53098d59224f admin admin] PATH_INFO: `/v3/users/4a953762447e42cfa82713049dc178a3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:23.908255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2691df-54d5-4234-bf06-53098d59224f admin admin] RBAC: Authorizing `identity:get_user(user_id=4a953762447e42cfa82713049dc178a3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:23.910126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2691df-54d5-4234-bf06-53098d59224f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:23.911697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Fri Jun 17 11:24:23 2022] GET /identity/v3/users/4a953762447e42cfa82713049dc178a3 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:26.079149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d6a6-9b2a-4f73-a5eb-c6f22126e016 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.079656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d6a6-9b2a-4f73-a5eb-c6f22126e016 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.079756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d6a6-9b2a-4f73-a5eb-c6f22126e016 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.081075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:26.086086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9db77b-8537-4fb0-9e33-963d247643c2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.086470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9db77b-8537-4fb0-9e33-963d247643c2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.086795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9db77b-8537-4fb0-9e33-963d247643c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.104570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4a9db77b-8537-4fb0-9e33-963d247643c2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:26.105770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4a9db77b-8537-4fb0-9e33-963d247643c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:26.107203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1a88-df1c-4bce-8b23-348a2a6334a9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.107498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1a88-df1c-4bce-8b23-348a2a6334a9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.107780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b1a88-df1c-4bce-8b23-348a2a6334a9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.109158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:24:26.114380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.114667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.114944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.155821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG passlib.handlers.bcrypt [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80103) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 17 11:24:26.156244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG passlib.handlers.bcrypt [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80103) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 17 11:24:26.190564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:26.202913 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:26.204524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:26.213206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9db77b-8537-4fb0-9e33-963d247643c2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.236955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:26.304885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:24:26.317195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9db77b-8537-4fb0-9e33-963d247643c2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.326784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.350871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.351599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.351884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.352159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.363325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:26.374551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:26.375502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:24:26.376310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:24:26.381396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.429755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc47a3b-d232-4f14-9107-c68422b4606e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.432061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:26.463939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:26.467327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.468764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.469062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.469355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.483831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123af771-3ef2-4b73-9c99-5b6c1163ab47 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.518783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.519453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.519738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.520022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.520771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:26.521081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:26.522510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:26.534547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:26.538794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:26.540324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:26.540441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.643066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:24:26.644795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788020e-8d6d-4d2c-92e6-4ef485f0babb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.648726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.651422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:26.680147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.681053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.681339 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.681613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] PATH_INFO: `/v3/users/4a953762447e42cfa82713049dc178a3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.682857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] RBAC: Authorizing `identity:get_user(user_id=4a953762447e42cfa82713049dc178a3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:26.684305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:26.685949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/users/4a953762447e42cfa82713049dc178a3 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:26.691195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.705551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:26.747845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300af21-2a46-4824-a6ea-54409520174c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.783330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.784123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.784405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.784684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.790460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd45c-54b6-4754-aaad-9ca0b8cb1d6d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.803070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:26.804631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:26.823668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.824460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.824743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.825017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:24:26.825354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:26.826189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:26.827162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:24:26.827162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:24:26.827162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:24:26.827162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:26.827382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:26.827470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:24:26.828419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:26.830022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:26.832803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.835486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:26.887681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:26.914771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:26.933547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdca341-883e-486f-9feb-eafd5dbd1202 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.934076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ace0-0001-4fb3-b5a5-0ada0a368891 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:26.966851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b1845-3646-48b3-a428-cd51be4adbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.967269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144cf153-b7be-472f-bd26-9d486c8eccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:26.967515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1845-3646-48b3-a428-cd51be4adbfc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.967797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1845-3646-48b3-a428-cd51be4adbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.967927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cf153-b7be-472f-bd26-9d486c8eccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:26.968071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1845-3646-48b3-a428-cd51be4adbfc None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.968217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cf153-b7be-472f-bd26-9d486c8eccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:26.968501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cf153-b7be-472f-bd26-9d486c8eccd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:26.968792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b1845-3646-48b3-a428-cd51be4adbfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:26.969101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b1845-3646-48b3-a428-cd51be4adbfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:26.969341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144cf153-b7be-472f-bd26-9d486c8eccd5 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.domain.id=default, user.id=4a953762447e42cfa82713049dc178a3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:26.969659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144cf153-b7be-472f-bd26-9d486c8eccd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:26.970515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b1845-3646-48b3-a428-cd51be4adbfc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:26.971128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144cf153-b7be-472f-bd26-9d486c8eccd5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:26.977801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/role_assignments?user.id=4a953762447e42cfa82713049dc178a3&scope.domain.id=default&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:26.989510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:29.641195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ffc4d-555a-47ac-abfa-6eb0633e8db3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:29.642170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ffc4d-555a-47ac-abfa-6eb0633e8db3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:29.642674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ffc4d-555a-47ac-abfa-6eb0633e8db3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:29.644262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:29.649865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898db393-3d38-48fb-8c97-ef060d72323e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:29.650815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898db393-3d38-48fb-8c97-ef060d72323e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:29.651289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898db393-3d38-48fb-8c97-ef060d72323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:29.679019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-898db393-3d38-48fb-8c97-ef060d72323e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:29.680607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-898db393-3d38-48fb-8c97-ef060d72323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:29.771427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a695c89-19cc-45a3-b7ca-37f1a88b7c6d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:29.771956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 11/23] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:29.772280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a695c89-19cc-45a3-b7ca-37f1a88b7c6d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:29.772775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a695c89-19cc-45a3-b7ca-37f1a88b7c6d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:29.774385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:29.786996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbecc4-2d80-405d-857f-01b2a5e403ff None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:29.787296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbecc4-2d80-405d-857f-01b2a5e403ff None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:29.787581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbecc4-2d80-405d-857f-01b2a5e403ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:29.819569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-40fbecc4-2d80-405d-857f-01b2a5e403ff None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:29.820774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-40fbecc4-2d80-405d-857f-01b2a5e403ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:29.826430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a695c89-19cc-45a3-b7ca-37f1a88b7c6d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:29.885449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:29.959290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 12/25] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:24:30.002065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbecc4-2d80-405d-857f-01b2a5e403ff None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:30.056409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a695c89-19cc-45a3-b7ca-37f1a88b7c6d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:30.105790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86697c2-a0d5-466c-be4e-e9063a66a681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:30.106529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86697c2-a0d5-466c-be4e-e9063a66a681 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.106819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86697c2-a0d5-466c-be4e-e9063a66a681 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.107102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86697c2-a0d5-466c-be4e-e9063a66a681 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.107913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86697c2-a0d5-466c-be4e-e9063a66a681 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:30.109906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86697c2-a0d5-466c-be4e-e9063a66a681 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:30.151487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbecc4-2d80-405d-857f-01b2a5e403ff None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:30.189286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdd606-0380-441a-8d25-86be310053ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:30.190391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdd606-0380-441a-8d25-86be310053ea None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.190822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdd606-0380-441a-8d25-86be310053ea None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.191245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdd606-0380-441a-8d25-86be310053ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.203546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdd606-0380-441a-8d25-86be310053ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-dbfdd606-0380-441a-8d25-86be310053ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:30.213038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:30.214168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:30.215132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:30.217463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 13/26] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:24:30.223025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b65544-9e5f-492b-9368-95d338914bc0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.223476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b65544-9e5f-492b-9368-95d338914bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.223901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b65544-9e5f-492b-9368-95d338914bc0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.225427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 14/27] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:30.230553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25dd75-fad6-4d79-8f75-861b6cca60be None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.231161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25dd75-fad6-4d79-8f75-861b6cca60be None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.231594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25dd75-fad6-4d79-8f75-861b6cca60be None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.233198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 15/28] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:30.234677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25dd75-fad6-4d79-8f75-861b6cca60be None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:30.262929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 14/29] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Jun 17 11:24:29 2022] POST /identity/v3/roles => generated 219 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:24:30.265194 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765b00-619c-482b-83c8-7cc83c1c4ac8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.265502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765b00-619c-482b-83c8-7cc83c1c4ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.265798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765b00-619c-482b-83c8-7cc83c1c4ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.274211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:30.286792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:30.377938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25dd75-fad6-4d79-8f75-861b6cca60be None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:30.423669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf588d-16c7-475e-979b-e93f18f6ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:30.424278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf588d-16c7-475e-979b-e93f18f6ff53 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.424569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf588d-16c7-475e-979b-e93f18f6ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.424852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf588d-16c7-475e-979b-e93f18f6ff53 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.425616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf588d-16c7-475e-979b-e93f18f6ff53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:30.425965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf588d-16c7-475e-979b-e93f18f6ff53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:30.427373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf588d-16c7-475e-979b-e93f18f6ff53 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:30.430671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ea765b00-619c-482b-83c8-7cc83c1c4ac8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:30.432520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ea765b00-619c-482b-83c8-7cc83c1c4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:30.440808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 16/30] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:30.442807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd532-2991-4b9e-b3b3-102baed6f255 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.442807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd532-2991-4b9e-b3b3-102baed6f255 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.442807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd532-2991-4b9e-b3b3-102baed6f255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.460425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fe8cd532-2991-4b9e-b3b3-102baed6f255 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:30.461658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fe8cd532-2991-4b9e-b3b3-102baed6f255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:30.513622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 15/31] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:30.514898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765b00-619c-482b-83c8-7cc83c1c4ac8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:30.536457 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 17/32] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:30.568258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd532-2991-4b9e-b3b3-102baed6f255 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:30.613677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:30.655919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765b00-619c-482b-83c8-7cc83c1c4ac8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:30.703807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:30.704783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.705173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.705556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] PATH_INFO: `/v3/users/4a953762447e42cfa82713049dc178a3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.707000 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=4a953762447e42cfa82713049dc178a3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:30.708701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:30.709573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd532-2991-4b9e-b3b3-102baed6f255 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:30.710576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 16/33] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:30 2022] GET /identity/v3/users/4a953762447e42cfa82713049dc178a3 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:30.711417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:30.714234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:30.748450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02adcee-ddcc-4069-b8ce-52a1d421863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:30.749426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02adcee-ddcc-4069-b8ce-52a1d421863f None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.749898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02adcee-ddcc-4069-b8ce-52a1d421863f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.750321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02adcee-ddcc-4069-b8ce-52a1d421863f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.751231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02adcee-ddcc-4069-b8ce-52a1d421863f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:30.752787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02adcee-ddcc-4069-b8ce-52a1d421863f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:30.820815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e3a8f-4736-41fa-8973-6a7e953a9fc0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:30.857888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d3c9e-1b75-4444-8a5d-b5875cbc056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:30.858719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d3c9e-1b75-4444-8a5d-b5875cbc056d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.859004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d3c9e-1b75-4444-8a5d-b5875cbc056d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.859278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d3c9e-1b75-4444-8a5d-b5875cbc056d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.870169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d3c9e-1b75-4444-8a5d-b5875cbc056d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:30.871623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d3c9e-1b75-4444-8a5d-b5875cbc056d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:30.883398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 18/34] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Jun 17 11:24:30 2022] POST /identity/v3/roles => generated 225 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:30.883398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f3e71-be48-4687-a250-9bc6f8b19177 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.883398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f3e71-be48-4687-a250-9bc6f8b19177 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.883398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f3e71-be48-4687-a250-9bc6f8b19177 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.883398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 19/35] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:30.883398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f3e71-be48-4687-a250-9bc6f8b19177 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-071d3c9e-1b75-4444-8a5d-b5875cbc056d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:24:30.898428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:30.899324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:30.900111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:24:30.900111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:24:30.900111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:24:30.900111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:24:30.900111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 17/36] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:30.901175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837caff-98c4-4caf-aaf5-c11cb766384c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:30.901881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837caff-98c4-4caf-aaf5-c11cb766384c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:30.901881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837caff-98c4-4caf-aaf5-c11cb766384c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:30.921193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5837caff-98c4-4caf-aaf5-c11cb766384c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:30.922589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5837caff-98c4-4caf-aaf5-c11cb766384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:30.984083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:31.010037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 18/37] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:31.010037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837caff-98c4-4caf-aaf5-c11cb766384c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:31.141010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837caff-98c4-4caf-aaf5-c11cb766384c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:31.163705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:31.178742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f3e71-be48-4687-a250-9bc6f8b19177 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:31.183898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:31.184574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.184861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.185142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.186311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:31.187153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:31.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:31.208024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 19/38] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:31.208937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:31.220031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80329d55-e464-420e-a92c-b8e4967f8b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:31.220840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80329d55-e464-420e-a92c-b8e4967f8b98 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.221128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80329d55-e464-420e-a92c-b8e4967f8b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.221407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80329d55-e464-420e-a92c-b8e4967f8b98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.222955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80329d55-e464-420e-a92c-b8e4967f8b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:31.224534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80329d55-e464-420e-a92c-b8e4967f8b98 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:31.230742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 20/39] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:31.230742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70dbd-c11b-461f-ac84-fec90e7501b1 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.230742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70dbd-c11b-461f-ac84-fec90e7501b1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.230742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70dbd-c11b-461f-ac84-fec90e7501b1 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.230742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 21/40] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:31.230742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70dbd-c11b-461f-ac84-fec90e7501b1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:31.261206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:31.315292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428776e5-b310-48a3-9949-9ed81c27da50 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:31.336611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70dbd-c11b-461f-ac84-fec90e7501b1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:31.350032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:31.350711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.351000 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.351280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.352041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:31.353452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:31.380949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 20/41] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Jun 17 11:24:31 2022] POST /identity/v3/roles => generated 223 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:31.380949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:31.381227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d19980-e4ce-4f0e-b1ee-4df7f84dbb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:31.381227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d19980-e4ce-4f0e-b1ee-4df7f84dbb89 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.381227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d19980-e4ce-4f0e-b1ee-4df7f84dbb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.381227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d19980-e4ce-4f0e-b1ee-4df7f84dbb89 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.381549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d19980-e4ce-4f0e-b1ee-4df7f84dbb89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:31.381549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d19980-e4ce-4f0e-b1ee-4df7f84dbb89 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:31.409711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 22/42] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:24:31 2022] POST /identity/v3/projects => generated 301 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:31.411246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd967b-b25a-42c7-ae33-f25fe3f0b4f6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.411643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd967b-b25a-42c7-ae33-f25fe3f0b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.412028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd967b-b25a-42c7-ae33-f25fe3f0b4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.441994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b0fd967b-b25a-42c7-ae33-f25fe3f0b4f6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:31.444038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b0fd967b-b25a-42c7-ae33-f25fe3f0b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:31.447354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:31.513023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b247f-b97e-4913-a1b6-36386b77ae33 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:31.516069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:31.545420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 23/43] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:31.561078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:31.562698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.562992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.563272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] PATH_INFO: `/v3/domains/default/users/4a953762447e42cfa82713049dc178a3/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.565535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4a953762447e42cfa82713049dc178a3, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:31.567060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:31.570170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd967b-b25a-42c7-ae33-f25fe3f0b4f6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:31.584658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 21/44] 10.222.0.25 () {58 vars in 1361 bytes} [Fri Jun 17 11:24:31 2022] PUT /identity/v3/domains/default/users/4a953762447e42cfa82713049dc178a3/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:24:31.700427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd967b-b25a-42c7-ae33-f25fe3f0b4f6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:31.736241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe4736-ece9-4919-b35c-b4e41f2c05bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:31.737066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe4736-ece9-4919-b35c-b4e41f2c05bb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.737358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe4736-ece9-4919-b35c-b4e41f2c05bb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.737643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe4736-ece9-4919-b35c-b4e41f2c05bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.738925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe4736-ece9-4919-b35c-b4e41f2c05bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:31.740403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe4736-ece9-4919-b35c-b4e41f2c05bb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:31.742095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 24/45] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:31.748441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:31.849367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428278-74ee-464a-a8a9-07d0999bb91e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:31.882971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009366f-2695-40a0-ae63-6612b1acb4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:31.883645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009366f-2695-40a0-ae63-6612b1acb4ea None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:31.883937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009366f-2695-40a0-ae63-6612b1acb4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:31.884222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009366f-2695-40a0-ae63-6612b1acb4ea None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:31.884963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009366f-2695-40a0-ae63-6612b1acb4ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:31.886497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009366f-2695-40a0-ae63-6612b1acb4ea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:31.912884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 22/46] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:24:31 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:33.700917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046f2c-ef5e-4276-8d8c-67c3e64f2c95 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:33.701410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046f2c-ef5e-4276-8d8c-67c3e64f2c95 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:33.701491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046f2c-ef5e-4276-8d8c-67c3e64f2c95 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:33.703432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 25/47] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:33.708886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fc0d2-fdb1-4008-aa8b-46e9574f5291 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:33.709257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fc0d2-fdb1-4008-aa8b-46e9574f5291 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:33.709571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fc0d2-fdb1-4008-aa8b-46e9574f5291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:33.726772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-426fc0d2-fdb1-4008-aa8b-46e9574f5291 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:33.727986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-426fc0d2-fdb1-4008-aa8b-46e9574f5291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:33.738946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:33.795982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 23/48] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:33.818787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046f2c-ef5e-4276-8d8c-67c3e64f2c95 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:33.828448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:33.922831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046f2c-ef5e-4276-8d8c-67c3e64f2c95 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:33.956329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77351d-0203-4836-bfcb-a0cd7c0f645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:33.957129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77351d-0203-4836-bfcb-a0cd7c0f645f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:33.957420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77351d-0203-4836-bfcb-a0cd7c0f645f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:33.957699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77351d-0203-4836-bfcb-a0cd7c0f645f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:33.968327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77351d-0203-4836-bfcb-a0cd7c0f645f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1d77351d-0203-4836-bfcb-a0cd7c0f645f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:33.976117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:33.977145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:33.977999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 26/49] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:33.983569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fc0d2-fdb1-4008-aa8b-46e9574f5291 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:34.046510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9dca-9710-42e3-bd97-d8bb95b169ca None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.046814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9dca-9710-42e3-bd97-d8bb95b169ca None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.047069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9dca-9710-42e3-bd97-d8bb95b169ca None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.048585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 27/50] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:24:34.053278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c8380-4ec3-48a7-ba76-d23854a06d68 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.053575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c8380-4ec3-48a7-ba76-d23854a06d68 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.053882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c8380-4ec3-48a7-ba76-d23854a06d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.071500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f34c8380-4ec3-48a7-ba76-d23854a06d68 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:34.072713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f34c8380-4ec3-48a7-ba76-d23854a06d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:34.086432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fc0d2-fdb1-4008-aa8b-46e9574f5291 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:34.120018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:34.120698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.120989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.121271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.122054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:34.122363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:34.123753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:34.131844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:34.132255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 24/51] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:34.137726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:34.141083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 28/52] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:24:34.148880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:34.164527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c8380-4ec3-48a7-ba76-d23854a06d68 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:34.236208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d955a-3f24-4913-9670-2160330d97a4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:34.263045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c8380-4ec3-48a7-ba76-d23854a06d68 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:34.271384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:34.272382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.273120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.273513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] PATH_INFO: `/v3/users/4a953762447e42cfa82713049dc178a3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.275138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] RBAC: Authorizing `identity:get_user(user_id=4a953762447e42cfa82713049dc178a3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:34.276650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:34.278344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 25/53] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:34 2022] GET /identity/v3/users/4a953762447e42cfa82713049dc178a3 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:34.283836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:34.297264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:34.298091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.298378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.298651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.299811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:34.301250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:34.302900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 29/54] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:34.308238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:34.360882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:34.374944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:34.384506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09255571-e5d1-4f2c-a8c1-7172c8ad4f98 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:34.412490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7736572-7638-4401-a9cb-a565b7996ee6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:34.418129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:34.418933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.419220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.419504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.420692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:34.422713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:34.424338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 26/55] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:34.429931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:34.447287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27926bbb-45db-485c-bc85-e339bb69b48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:34.447959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27926bbb-45db-485c-bc85-e339bb69b48b None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.448248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27926bbb-45db-485c-bc85-e339bb69b48b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.448530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27926bbb-45db-485c-bc85-e339bb69b48b None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.449247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27926bbb-45db-485c-bc85-e339bb69b48b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:34.450729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27926bbb-45db-485c-bc85-e339bb69b48b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:34.474841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 30/56] 10.222.0.25 () {60 vars in 1140 bytes} [Fri Jun 17 11:24:34 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:24:34.532516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9898b1-2fff-4598-982e-a23286e4bc3e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:34.569012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6992c51-de00-43cd-894d-11e419330958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:34.569702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6992c51-de00-43cd-894d-11e419330958 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:34.570031 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6992c51-de00-43cd-894d-11e419330958 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:34.570318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6992c51-de00-43cd-894d-11e419330958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:34.571175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6992c51-de00-43cd-894d-11e419330958 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.domain.id=default, user.id=4a953762447e42cfa82713049dc178a3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:34.571498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6992c51-de00-43cd-894d-11e419330958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:34.572967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6992c51-de00-43cd-894d-11e419330958 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:34.579859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 27/57] 10.222.0.25 () {56 vars in 1322 bytes} [Fri Jun 17 11:24:34 2022] GET /identity/v3/role_assignments?user.id=4a953762447e42cfa82713049dc178a3&scope.domain.id=default&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:36.615710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c7c7f-ef96-4c13-8b30-ef602be7fea4 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:36.616242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c7c7f-ef96-4c13-8b30-ef602be7fea4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:36.616339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c7c7f-ef96-4c13-8b30-ef602be7fea4 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:36.617663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 31/58] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:36.622894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee169cb-99fb-4566-abe2-82e92a0b2006 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:36.623284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee169cb-99fb-4566-abe2-82e92a0b2006 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:36.623467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee169cb-99fb-4566-abe2-82e92a0b2006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:36.641127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5ee169cb-99fb-4566-abe2-82e92a0b2006 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:36.642375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5ee169cb-99fb-4566-abe2-82e92a0b2006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:36.644311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:36.711280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 28/59] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:36.736209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c7c7f-ef96-4c13-8b30-ef602be7fea4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:36.844638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c7c7f-ef96-4c13-8b30-ef602be7fea4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:36.860772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:36.879160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801872ce-10d6-4fd6-a420-0a06664b5136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:36.879907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801872ce-10d6-4fd6-a420-0a06664b5136 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:36.880199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801872ce-10d6-4fd6-a420-0a06664b5136 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:36.880484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801872ce-10d6-4fd6-a420-0a06664b5136 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:36.881669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801872ce-10d6-4fd6-a420-0a06664b5136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:36.883157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801872ce-10d6-4fd6-a420-0a06664b5136 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:36.884788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 32/60] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:36.890444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee169cb-99fb-4566-abe2-82e92a0b2006 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:36.985982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee169cb-99fb-4566-abe2-82e92a0b2006 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:37.018734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0f900-0b22-4da5-b4ae-82c0197f9af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:37.019402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f900-0b22-4da5-b4ae-82c0197f9af4 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:37.019691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f900-0b22-4da5-b4ae-82c0197f9af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:37.019969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0f900-0b22-4da5-b4ae-82c0197f9af4 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:37.020678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0f900-0b22-4da5-b4ae-82c0197f9af4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:37.022165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0f900-0b22-4da5-b4ae-82c0197f9af4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:37.035705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:37.048256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 29/61] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:24:36 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:40.285007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43008eb2-d1bb-4285-b2c7-35b205b7dc7d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.286055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43008eb2-d1bb-4285-b2c7-35b205b7dc7d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.286551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43008eb2-d1bb-4285-b2c7-35b205b7dc7d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.288159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 33/62] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:40.293766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cd1b8-03ce-4569-b3cf-6492b9354b08 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.294622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cd1b8-03ce-4569-b3cf-6492b9354b08 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.295118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cd1b8-03ce-4569-b3cf-6492b9354b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.297495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3789-ece4-40ef-a205-15dddb39e20e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.298554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3789-ece4-40ef-a205-15dddb39e20e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.299614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3789-ece4-40ef-a205-15dddb39e20e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.304753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 34/63] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:40.307934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d1839-b817-4c0a-b9db-248585dfcea5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.308436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d1839-b817-4c0a-b9db-248585dfcea5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.308914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d1839-b817-4c0a-b9db-248585dfcea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.332027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6a8cd1b8-03ce-4569-b3cf-6492b9354b08 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:40.333666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6a8cd1b8-03ce-4569-b3cf-6492b9354b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:40.343065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b17d1839-b817-4c0a-b9db-248585dfcea5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:40.349308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b17d1839-b817-4c0a-b9db-248585dfcea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:40.464145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 30/64] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:40.465324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fc42a-7e3e-42c9-810b-b1825801cb63 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.465620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fc42a-7e3e-42c9-810b-b1825801cb63 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.465930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fc42a-7e3e-42c9-810b-b1825801cb63 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.467212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 31/65] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:40.470178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 35/66] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:40.475365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8cc0-44d2-4f72-9b18-8af2974a8760 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.475663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8cc0-44d2-4f72-9b18-8af2974a8760 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.475950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e8cc0-44d2-4f72-9b18-8af2974a8760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.488804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d1839-b817-4c0a-b9db-248585dfcea5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:40.493666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2b9e8cc0-44d2-4f72-9b18-8af2974a8760 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:40.495048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2b9e8cc0-44d2-4f72-9b18-8af2974a8760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:40.568687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 32/67] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:24:40.569806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8cc0-44d2-4f72-9b18-8af2974a8760 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:40.598258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d1839-b817-4c0a-b9db-248585dfcea5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:40.614531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:40.630080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:40.644821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:40.645641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.646028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.646505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.659174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:40.667578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:40.668495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:40.669247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:40.676874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 36/68] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:40.678047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:40.679892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e8cc0-44d2-4f72-9b18-8af2974a8760 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:40.716758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:40.717674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.718247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.718783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.730448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:40.741308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:40.742425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:40.743169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:24:40.746394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 33/69] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:40.747439 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:40.788501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac702cb5-587a-46c1-93cf-64f41f58f19a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:40.825163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:40.826371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.826819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.827372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.838877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:40.848509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:40.849443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:40.850210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:40.853160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 37/70] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:24:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:40.854370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:40.862914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161d231-6247-4aff-890c-6218d80faa22 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:40.879609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:40.897556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:40.903387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:40.904162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:40.905320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:40.906252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:40.907091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:40.907478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:40.909055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:40.918599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 34/71] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:40.919517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:40.972614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819c8f6-2442-46ee-b7b9-0f92e79c914a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.010004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0e0ab-de87-4e58-b4b8-8601684f395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.010688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0e0ab-de87-4e58-b4b8-8601684f395b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.010971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0e0ab-de87-4e58-b4b8-8601684f395b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.011251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0e0ab-de87-4e58-b4b8-8601684f395b None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.012008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0e0ab-de87-4e58-b4b8-8601684f395b None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:41.012319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0e0ab-de87-4e58-b4b8-8601684f395b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:41.013704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0e0ab-de87-4e58-b4b8-8601684f395b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:41.021989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 38/72] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:41.023242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65d674-4e59-4d55-a032-38e972de3331 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.023534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65d674-4e59-4d55-a032-38e972de3331 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.023815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65d674-4e59-4d55-a032-38e972de3331 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.025215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 39/73] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:41.025760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65d674-4e59-4d55-a032-38e972de3331 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.044875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379087e-9371-4201-8566-add9369fb4e8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.079004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.079603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.079891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.080171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.080921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:41.081234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:41.082664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:41.090671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 35/74] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:24:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:41.091454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.128731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.133284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.136786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65d674-4e59-4d55-a032-38e972de3331 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.172991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041d03d-986b-4a30-a024-82c4297dcdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.173908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041d03d-986b-4a30-a024-82c4297dcdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.174198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041d03d-986b-4a30-a024-82c4297dcdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.174843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041d03d-986b-4a30-a024-82c4297dcdc8 None admin] PATH_INFO: `/v3/users/4a953762447e42cfa82713049dc178a3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.176298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041d03d-986b-4a30-a024-82c4297dcdc8 None admin] RBAC: Authorizing `identity:get_user(user_id=4a953762447e42cfa82713049dc178a3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:41.177745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041d03d-986b-4a30-a024-82c4297dcdc8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:41.179420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 40/75] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/users/4a953762447e42cfa82713049dc178a3 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:41.180561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b073b-074f-4f65-9bca-04a8709500e0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.180854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b073b-074f-4f65-9bca-04a8709500e0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.181137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b073b-074f-4f65-9bca-04a8709500e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.196393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1956e98-29d0-44e2-9978-a2319c5733b6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.198839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-196b073b-074f-4f65-9bca-04a8709500e0 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:41.200055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-196b073b-074f-4f65-9bca-04a8709500e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:41.231505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec40b2-9f95-4763-b6d9-eb65f730efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.232889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec40b2-9f95-4763-b6d9-eb65f730efe4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.233388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec40b2-9f95-4763-b6d9-eb65f730efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.233962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec40b2-9f95-4763-b6d9-eb65f730efe4 None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.251108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec40b2-9f95-4763-b6d9-eb65f730efe4 None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:41.252604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec40b2-9f95-4763-b6d9-eb65f730efe4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:41.254387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 36/76] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:41.255656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d228f-d9d0-4a7d-9732-ad9b3057f56a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.255945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d228f-d9d0-4a7d-9732-ad9b3057f56a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.256226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d228f-d9d0-4a7d-9732-ad9b3057f56a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.257478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 37/77] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:41.258206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d228f-d9d0-4a7d-9732-ad9b3057f56a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.284451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 41/78] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:41.285315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b073b-074f-4f65-9bca-04a8709500e0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.362234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.367598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d228f-d9d0-4a7d-9732-ad9b3057f56a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.397523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.405577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b073b-074f-4f65-9bca-04a8709500e0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.409681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.410599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.410891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.411172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.412377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:41.414118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:41.415755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 38/79] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:41.416530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.444266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7118f98-4291-451d-8977-511e334d3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.444937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7118f98-4291-451d-8977-511e334d3b98 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.445227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7118f98-4291-451d-8977-511e334d3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.445538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7118f98-4291-451d-8977-511e334d3b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.516815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7118f98-4291-451d-8977-511e334d3b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:41.518358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7118f98-4291-451d-8977-511e334d3b98 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:41.533397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24c8e-ca39-4c01-ab6b-a0606f518fac None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.570378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.571035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.571319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.571594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.606967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.616778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.630968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 42/80] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:41.630968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f29844-28f0-4e3c-8ff1-3f29768ca2fc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.630968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f29844-28f0-4e3c-8ff1-3f29768ca2fc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.630968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f29844-28f0-4e3c-8ff1-3f29768ca2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.649144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-39f29844-28f0-4e3c-8ff1-3f29768ca2fc None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:41.650377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:41.650565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-39f29844-28f0-4e3c-8ff1-3f29768ca2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:41.651863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:41.719246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 43/81] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:41.720343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f29844-28f0-4e3c-8ff1-3f29768ca2fc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.777984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 39/82] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:41.779464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.823292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f29844-28f0-4e3c-8ff1-3f29768ca2fc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.846906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.847694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:41.860388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.861206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.861509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.861794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.872141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:41.881292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:41.882234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:41.883146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 44/83] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:41.884262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:41.894264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010b48d-2b70-418e-8de3-fd70370cf835 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:41.932902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:41.933566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:41.933882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:41.934164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:41.992980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84119141-7948-4720-8aba-c4db25f2b158 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.001811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.003312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.027055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.027924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.028214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.028495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.029738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.031333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.033229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 45/84] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.033867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.067039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.089101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.106026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 40/85] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:42.106871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.138220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5275f-a409-471c-aa90-33306fcf8467 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.170716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.172036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.172320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.172675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.173869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.175325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.177021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 46/86] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.177785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.210759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3613dc-d5c2-47a2-b85a-a1a46cd1a882 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.244361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.245223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.245508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.245785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.246993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.248451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.250749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 41/87] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.251510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.280895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77e4f6-b9bc-46ef-ad14-7eb0cc1fa2a3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.281749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.301832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.316061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.316731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.317016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.317291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.318062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:42.318375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.319754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.327810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 47/88] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.328505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.355524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aca51-ea1d-4819-b28c-b1fdd4fb505c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.389445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.390106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.390398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.390678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.391518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=4a953762447e42cfa82713049dc178a3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:42.391839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.393281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.398683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 42/89] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/role_assignments?user.id=4a953762447e42cfa82713049dc178a3&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.399448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.431531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3163-88df-47b7-90a1-55461aa11fd0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.472459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.473899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.474191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.474742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.476216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.478380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.481191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 48/90] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.481817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.516271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.523265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e79c5-8afc-480d-b95c-222279ada2ec None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.541287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.562609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.563304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.563598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.563868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.564610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.566179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.587780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 43/91] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:24:42 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:42.588617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.601890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561f7f-e88a-4886-aa7d-9ef7347c6bc8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.670204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.670817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.671141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.671421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.672283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=4466175761d84340a500132eb2b738d2) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:42.672601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.674150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.684686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 49/92] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/role_assignments?user.id=4466175761d84340a500132eb2b738d2&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=9ebaaf52f4cd402880b403c50600168e => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.685556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.744930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b357a-f7d7-4b52-9db6-6b8ad93df1e6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.787137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.788673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.789221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.789824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.791515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.793534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.795662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 44/93] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.802429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc55d59-b8cb-4319-b807-c20e11cecc6b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.804603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:42.819058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.846354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:42.846861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.847695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.848432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.848889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:42.850153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] RBAC: Adding query filter params (role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=4466175761d84340a500132eb2b738d2) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:42.851006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:42.853141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:42.861373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 50/94] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/role_assignments?user.id=4466175761d84340a500132eb2b738d2&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:42.941038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1f37-7eb4-4128-8579-2ce2b60ff661 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:42.986577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:42.987576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:42.988043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:42.988496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:43.067217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:43.069001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:43.095371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:43.174594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 45/95] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:43.181289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:43.298852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7d52-4706-47fa-8f11-c578749f92e7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:43.339454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06cba-3b48-41ef-bfe9-2c18f08ce519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:43.340332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06cba-3b48-41ef-bfe9-2c18f08ce519 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:43.340620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06cba-3b48-41ef-bfe9-2c18f08ce519 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:43.340902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06cba-3b48-41ef-bfe9-2c18f08ce519 None admin] PATH_INFO: `/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:43.342137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06cba-3b48-41ef-bfe9-2c18f08ce519 None admin] RBAC: Authorizing `identity:get_project(project_id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:43.343617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06cba-3b48-41ef-bfe9-2c18f08ce519 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:43.345369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 51/96] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:43 2022] GET /identity/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:43.352242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:43.457084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba38921-2a10-43dd-91ae-6013d99a3aa6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:43.478309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:43.493318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff5803-5553-442b-80a6-29111233ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:43.494020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff5803-5553-442b-80a6-29111233ceed None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:43.494311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff5803-5553-442b-80a6-29111233ceed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:43.494594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff5803-5553-442b-80a6-29111233ceed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:43.495444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff5803-5553-442b-80a6-29111233ceed None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e, user.id=4466175761d84340a500132eb2b738d2) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:43.495770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff5803-5553-442b-80a6-29111233ceed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:43.497224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff5803-5553-442b-80a6-29111233ceed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:43.503957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 46/97] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:43 2022] GET /identity/v3/role_assignments?user.id=4466175761d84340a500132eb2b738d2&scope.project.id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e&role.id=9ebaaf52f4cd402880b403c50600168e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:45.100869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7471cfd-a2d8-4b67-9e9c-56f7213aad6b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.101372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7471cfd-a2d8-4b67-9e9c-56f7213aad6b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.101447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7471cfd-a2d8-4b67-9e9c-56f7213aad6b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.102856 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 52/98] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:45.108073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe8fe1-e8ce-499e-b342-c4b06d2f2fd8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.109009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe8fe1-e8ce-499e-b342-c4b06d2f2fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.109173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe8fe1-e8ce-499e-b342-c4b06d2f2fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.137935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-49fe8fe1-e8ce-499e-b342-c4b06d2f2fd8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:45.139160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-49fe8fe1-e8ce-499e-b342-c4b06d2f2fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:45.239829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 47/99] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:45.274440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7471cfd-a2d8-4b67-9e9c-56f7213aad6b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:45.342015 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:45.407581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7471cfd-a2d8-4b67-9e9c-56f7213aad6b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:45.453349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:45.454530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.455036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.455507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.469918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:45.477810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:45.478663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:45.479400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:45.482032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 53/100] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:45.494974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe8fe1-e8ce-499e-b342-c4b06d2f2fd8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:45.644658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe8fe1-e8ce-499e-b342-c4b06d2f2fd8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:45.669457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:45.705504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994eb51-0b50-482b-b5ef-9db11eed9da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:45.705504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994eb51-0b50-482b-b5ef-9db11eed9da5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.705504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994eb51-0b50-482b-b5ef-9db11eed9da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.705504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994eb51-0b50-482b-b5ef-9db11eed9da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.705937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994eb51-0b50-482b-b5ef-9db11eed9da5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:45.705937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994eb51-0b50-482b-b5ef-9db11eed9da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:45.707234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994eb51-0b50-482b-b5ef-9db11eed9da5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:45.722969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 48/101] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:45.728536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:45.813349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d19a-7cd0-41a4-b4f5-76e4830ad3bf None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.813662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d19a-7cd0-41a4-b4f5-76e4830ad3bf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.813960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d19a-7cd0-41a4-b4f5-76e4830ad3bf None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.815861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 49/102] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:45.821203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8feaa-1959-4433-93d2-c09714c2c5d6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.821512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8feaa-1959-4433-93d2-c09714c2c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.821980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8feaa-1959-4433-93d2-c09714c2c5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.850960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b9a8feaa-1959-4433-93d2-c09714c2c5d6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:45.852201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b9a8feaa-1959-4433-93d2-c09714c2c5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:45.901696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721576f-7c7e-4bc1-97c6-853116d3663f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:45.912264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:45.959569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f57ad-9772-4d39-a7dd-5bd79980136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:45.962469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f57ad-9772-4d39-a7dd-5bd79980136c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.963127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f57ad-9772-4d39-a7dd-5bd79980136c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.964527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f57ad-9772-4d39-a7dd-5bd79980136c None admin] PATH_INFO: `/v3/users/4a953762447e42cfa82713049dc178a3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.966130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f57ad-9772-4d39-a7dd-5bd79980136c None admin] RBAC: Authorizing `identity:get_user(user_id=4a953762447e42cfa82713049dc178a3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:45.970914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f57ad-9772-4d39-a7dd-5bd79980136c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:45.973881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 54/103] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:45 2022] GET /identity/v3/users/4a953762447e42cfa82713049dc178a3 => generated 247 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:45.976160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770fe3b-6636-4f01-8420-e7c82ae13367 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.977238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770fe3b-6636-4f01-8420-e7c82ae13367 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.979358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770fe3b-6636-4f01-8420-e7c82ae13367 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:45.983164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 50/104] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:45.984767 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8feaa-1959-4433-93d2-c09714c2c5d6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:45.985173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 55/105] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:45.988696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3a36-3755-47f8-b478-696da4b98770 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:45.989190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3a36-3755-47f8-b478-696da4b98770 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:45.989663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3a36-3755-47f8-b478-696da4b98770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.029632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-97cf3a36-3755-47f8-b478-696da4b98770 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:46.031726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-97cf3a36-3755-47f8-b478-696da4b98770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:46.150437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8feaa-1959-4433-93d2-c09714c2c5d6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.150942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 56/106] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:46.151617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3a36-3755-47f8-b478-696da4b98770 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.169852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:46.205120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e88881-faf5-4f91-bbde-fbbe235eab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.206043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e88881-faf5-4f91-bbde-fbbe235eab94 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.206331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e88881-faf5-4f91-bbde-fbbe235eab94 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.206603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e88881-faf5-4f91-bbde-fbbe235eab94 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.207814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e88881-faf5-4f91-bbde-fbbe235eab94 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:46.209272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e88881-faf5-4f91-bbde-fbbe235eab94 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:46.211300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 51/107] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:45 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:46.212371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164a824-3851-4ea3-8d7e-8501de49f3f7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.212658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164a824-3851-4ea3-8d7e-8501de49f3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.212937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164a824-3851-4ea3-8d7e-8501de49f3f7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.214262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 52/108] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:46.214966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164a824-3851-4ea3-8d7e-8501de49f3f7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.290918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3a36-3755-47f8-b478-696da4b98770 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.294270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:46.333664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676e435-a4f4-4134-a18d-a0c5dfe3e4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.334532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e435-a4f4-4134-a18d-a0c5dfe3e4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.334810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e435-a4f4-4134-a18d-a0c5dfe3e4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.335082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e435-a4f4-4134-a18d-a0c5dfe3e4d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.351060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e435-a4f4-4134-a18d-a0c5dfe3e4d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:46.359783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164a824-3851-4ea3-8d7e-8501de49f3f7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-c676e435-a4f4-4134-a18d-a0c5dfe3e4d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:46.363002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:46.363829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:46.364589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 57/109] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:24:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:46.365671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9923-67c3-4449-9ace-4b83b1540daa None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.368056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9923-67c3-4449-9ace-4b83b1540daa None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.368056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9923-67c3-4449-9ace-4b83b1540daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.390382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d8ec9923-67c3-4449-9ace-4b83b1540daa None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:46.392546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d8ec9923-67c3-4449-9ace-4b83b1540daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:46.402632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.403429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.403712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.403986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.416257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:46.424199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:46.425061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:24:46.425815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 53/110] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:46.426477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.466556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:46.481503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 58/111] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:46.482631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9923-67c3-4449-9ace-4b83b1540daa None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.554710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ba95c-068b-452a-8a69-f40a8c62853a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.594297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.595214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.595505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.595788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/4a953762447e42cfa82713049dc178a3/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.598017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] RBAC: Authorizing `identity:create_grant(project_id=577d26cace0e4d3fba244021e34a6d6b, user_id=4a953762447e42cfa82713049dc178a3, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:46.599370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:46.599608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:46.608277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9923-67c3-4449-9ace-4b83b1540daa None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.617887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 54/112] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:24:46 2022] PUT /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/4a953762447e42cfa82713049dc178a3/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:24:46.617887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.684981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.686034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.686515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.686970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.688641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:46.689129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:46.691333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:46.704796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 59/113] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:24:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:46.706064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.736605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:46.744675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c987a9-e1e3-489f-9f71-e863468dd34e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.783070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6340c9-7b33-4764-b657-150a816e6589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.783763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6340c9-7b33-4764-b657-150a816e6589 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.784039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6340c9-7b33-4764-b657-150a816e6589 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.784318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6340c9-7b33-4764-b657-150a816e6589 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.785091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6340c9-7b33-4764-b657-150a816e6589 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:46.785400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6340c9-7b33-4764-b657-150a816e6589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:46.786832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6340c9-7b33-4764-b657-150a816e6589 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:46.796725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 55/114] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:46.798142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648457eb-8e94-4bd1-8b0b-411da7dd7c74 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.798931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648457eb-8e94-4bd1-8b0b-411da7dd7c74 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.799222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648457eb-8e94-4bd1-8b0b-411da7dd7c74 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.804627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 56/115] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:46.804627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648457eb-8e94-4bd1-8b0b-411da7dd7c74 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.833434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857428-4395-432e-bc03-c66f250a1c88 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.873778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a3d7a-a672-4ec1-aa2a-f16490793392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.875125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3d7a-a672-4ec1-aa2a-f16490793392 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.875413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3d7a-a672-4ec1-aa2a-f16490793392 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.875690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3d7a-a672-4ec1-aa2a-f16490793392 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.876870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3d7a-a672-4ec1-aa2a-f16490793392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:46.878355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3d7a-a672-4ec1-aa2a-f16490793392 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:46.879973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 60/116] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:46.885704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0c40c-f634-4baa-93da-be85df1d01a1 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.886213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0c40c-f634-4baa-93da-be85df1d01a1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.886764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0c40c-f634-4baa-93da-be85df1d01a1 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.887903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 61/117] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:46.888545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0c40c-f634-4baa-93da-be85df1d01a1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:46.917421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:46.923257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648457eb-8e94-4bd1-8b0b-411da7dd7c74 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:46.962058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ee85-3d97-488c-88c9-ea3213cdc6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:46.962952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ee85-3d97-488c-88c9-ea3213cdc6fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.963244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ee85-3d97-488c-88c9-ea3213cdc6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.963531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ee85-3d97-488c-88c9-ea3213cdc6fb None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.964754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519ee85-3d97-488c-88c9-ea3213cdc6fb None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:46.966326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519ee85-3d97-488c-88c9-ea3213cdc6fb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:46.968040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 57/118] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:46 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:46.969206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ede8f-5622-4c54-ac1e-51e60124b0c8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:46.969497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ede8f-5622-4c54-ac1e-51e60124b0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:46.969782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ede8f-5622-4c54-ac1e-51e60124b0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:46.986047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f02ede8f-5622-4c54-ac1e-51e60124b0c8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:46.987312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f02ede8f-5622-4c54-ac1e-51e60124b0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:47.000893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0c40c-f634-4baa-93da-be85df1d01a1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.006155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:47.040558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.041534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.041876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.042353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.043600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:47.045163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:47.051754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 62/119] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:46 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:47.052430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.059761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 58/120] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:47.070067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c96c0-ed92-43b4-ba45-b93699316c6b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.070067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c96c0-ed92-43b4-ba45-b93699316c6b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.070067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c96c0-ed92-43b4-ba45-b93699316c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.079955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2d1c96c0-ed92-43b4-ba45-b93699316c6b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:47.081249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2d1c96c0-ed92-43b4-ba45-b93699316c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:47.151877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 59/121] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:47.152728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c96c0-ed92-43b4-ba45-b93699316c6b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.168798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:47.179542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd86d5a-a399-477a-89b0-d6f1cd6a87ef None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.218794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.219472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.219816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.220104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.220823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:47.222311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:47.252357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:47.261874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 63/122] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:24:47 2022] POST /identity/v3/projects => generated 302 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:47.261874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.282346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c96c0-ed92-43b4-ba45-b93699316c6b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.326782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.327879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.328354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.328740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.330233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:47.332191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:47.334487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 60/123] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:47 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:47.335290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.407486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7d66d-d34d-43a0-8458-818ff9e43070 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.453586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.454607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.454897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.455175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.457136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:47.458629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:47.460220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa30d61-8481-4bd2-988f-62bdfe669a83 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.460332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 64/124] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:47 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:47.461190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.477996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:47.513154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.514228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.514568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.514992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.532257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:47.543567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:47.544507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 61/125] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:47.545729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.586254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:47.587310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa409-8f5e-4c76-874f-b1a43e02fd0c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.638466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.640591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.641050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.641478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.656132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:47.664865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:47.665874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:47.666660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 65/126] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:47.667317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.681786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35d228-10db-4311-82cb-f153641f4359 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.719932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.721098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.721530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.721982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/4466175761d84340a500132eb2b738d2/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.724597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=577d26cace0e4d3fba244021e34a6d6b, user_id=4466175761d84340a500132eb2b738d2, role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:47.726477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:47.741809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:47.745298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 62/127] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:24:47 2022] PUT /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/4466175761d84340a500132eb2b738d2/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:24:47.746309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.775413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdee7c7-19a0-44cf-8af5-b086c57d985f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:47.814106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:47.815035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:47.815329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:47.815614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/4466175761d84340a500132eb2b738d2/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:47.817806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] RBAC: Authorizing `identity:create_grant(project_id=577d26cace0e4d3fba244021e34a6d6b, user_id=4466175761d84340a500132eb2b738d2, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:47.819338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:47.840962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:47.841212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 66/128] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:24:47 2022] PUT /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/4466175761d84340a500132eb2b738d2/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:24:47.845934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:47.928050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e26842-9a3d-43c1-a273-24b0c826a9b9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:48.030667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8c1bf-b557-4c82-acbc-0045323d076a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:48.031666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:48.032596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:48.033071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:48.033537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:48.034519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:48.035041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:48.036781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:48.060147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 63/129] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:48.069188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:48.102382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:48.111837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:48.112548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:48.112841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:48.113132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:48.114039 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:48.114367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:48.115806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:48.125892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 67/130] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:48.134874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:48.316321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4acaff-5501-439d-8148-e836454ad0d0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:48.317049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d331-d10b-492c-8ced-47829a9a0ba9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:48.382586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:48.383492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:48.383786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:48.384073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:48.385383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:48.386874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:48.388847 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 68/131] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:48 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:48.395288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:48.414967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:48.416631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:48.417216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:48.417792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] PATH_INFO: `/v3/users/2d7eac9126ed4f3d9c2cc54527bcaa3a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:48.421431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:48.459506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] RBAC: Authorizing `identity:get_user(user_id=2d7eac9126ed4f3d9c2cc54527bcaa3a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:48.462381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:48.465559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 64/132] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:48 2022] GET /identity/v3/users/2d7eac9126ed4f3d9c2cc54527bcaa3a => generated 289 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:48.474135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:48.502448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:48.577645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33c277-a547-47dd-9255-3cf303971dd9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:48.652258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:48.653235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:48.653524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:48.653802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] PATH_INFO: `/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:48.657949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] RBAC: Authorizing `identity:get_project(project_id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:48.659585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:48.665968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 69/133] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:48 2022] GET /identity/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:48.671259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:48.768689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dabd6c-8488-4cac-b18a-aaa196c6989b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:48.856100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:48.856699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:48.856992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:48.857279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:48.915967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a086474-1d08-4b58-8d17-4bd3083b781e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:48.981793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:48.991282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:49.002550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9bf47-7a7b-48c6-88ea-77acea304b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:49.003464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9bf47-7a7b-48c6-88ea-77acea304b5b None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:49.003751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9bf47-7a7b-48c6-88ea-77acea304b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:49.004036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9bf47-7a7b-48c6-88ea-77acea304b5b None admin] PATH_INFO: `/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e/users/4466175761d84340a500132eb2b738d2/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:49.006109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9bf47-7a7b-48c6-88ea-77acea304b5b None admin] RBAC: Authorizing `identity:create_grant(project_id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e, user_id=4466175761d84340a500132eb2b738d2, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:49.006509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:49.007553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9bf47-7a7b-48c6-88ea-77acea304b5b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:49.008304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:49.054205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 70/134] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:24:48 2022] PUT /identity/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e/users/4466175761d84340a500132eb2b738d2/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:24:49.097495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50475334-5c12-474f-b2f1-517c06a88371 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:49.097784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50475334-5c12-474f-b2f1-517c06a88371 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:49.098098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50475334-5c12-474f-b2f1-517c06a88371 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:49.103489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 71/135] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:49.104930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78967d-9712-4daa-8bbc-875b22816db5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:49.105225 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78967d-9712-4daa-8bbc-875b22816db5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:49.110037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78967d-9712-4daa-8bbc-875b22816db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:49.153386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1b78967d-9712-4daa-8bbc-875b22816db5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:49.154681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1b78967d-9712-4daa-8bbc-875b22816db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:49.205862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 65/136] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:24:49.214937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:49.304435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 72/137] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:49.333623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78967d-9712-4daa-8bbc-875b22816db5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:49.371586 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:49.382161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e5ea4-ce28-42e3-963d-842aeeabf5b8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:49.420493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:49.421743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:49.422459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:49.422942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:49.424371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:49.426078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:49.428408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 66/138] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:49 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:49.434654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:49.540765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe9783-7187-4f2b-bdbf-035b66752962 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:49.576229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:49.577827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:49.578346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:49.578877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:49.579919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] RBAC: Adding query filter params (role.id=21c858abff6f4b9c90786fbcec903396, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=2d7eac9126ed4f3d9c2cc54527bcaa3a) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:49.580458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:49.582203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:49.588341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 67/139] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:49 2022] GET /identity/v3/role_assignments?user.id=2d7eac9126ed4f3d9c2cc54527bcaa3a&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=21c858abff6f4b9c90786fbcec903396 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:49.596135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78967d-9712-4daa-8bbc-875b22816db5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:49.637979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:49.672562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:49.673385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:49.673674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:49.673982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:49.695287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:49.703544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:49.704441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:49.705221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 73/140] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:24:49.710895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:49.753934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:49.866936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc43a1-b88d-4972-b7b7-54e0c8bfbc8e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:49.914035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:49.914659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:49.914951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:49.915231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:49.916009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:49.916323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:49.917733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:49.927601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 68/141] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:49.933294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:50.066019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937baf6-36d1-4bab-aba0-f34a22752ca2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:50.115958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:50.116854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:50.117148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:50.117434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] PATH_INFO: `/v3/users/4a953762447e42cfa82713049dc178a3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:50.118776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] RBAC: Authorizing `identity:get_user(user_id=4a953762447e42cfa82713049dc178a3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:50.120249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:50.121942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 74/142] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:49 2022] GET /identity/v3/users/4a953762447e42cfa82713049dc178a3 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:50.127661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:50.266879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:50.279509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c5e92-e356-4940-a362-8715a6b22a21 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:50.327623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:50.328296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:50.328585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:50.328864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:50.415096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:50.416589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:50.518708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:50.540198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 69/143] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:50.545768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:50.633489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:50.692333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1f730-ced7-422d-a544-233d3175d77e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:50.737485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e5062-9f40-4390-b28a-52c1dc425bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:50.738397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e5062-9f40-4390-b28a-52c1dc425bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:50.738692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e5062-9f40-4390-b28a-52c1dc425bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:50.738978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e5062-9f40-4390-b28a-52c1dc425bb1 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:50.740213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e5062-9f40-4390-b28a-52c1dc425bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:50.741687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e5062-9f40-4390-b28a-52c1dc425bb1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:50.743663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 75/144] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:50 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:50.749823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:50.764294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b882e2-cb4f-47f6-85ed-ccf28b952b7d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:50.765128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b882e2-cb4f-47f6-85ed-ccf28b952b7d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:50.765415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b882e2-cb4f-47f6-85ed-ccf28b952b7d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:50.766973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 76/145] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:24:50.772862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494500-5301-4152-bc18-67f4ed639922 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:50.773105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494500-5301-4152-bc18-67f4ed639922 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:50.773387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494500-5301-4152-bc18-67f4ed639922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:50.800665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a5494500-5301-4152-bc18-67f4ed639922 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:50.802277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a5494500-5301-4152-bc18-67f4ed639922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:50.927252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 77/146] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:50.947557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95414-62ba-4315-964c-54c2ae8f114a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:50.973913 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5494500-5301-4152-bc18-67f4ed639922 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:51.003449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:51.004128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.004411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.004686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.005522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=4a953762447e42cfa82713049dc178a3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:51.005855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:51.007302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:51.018400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 70/147] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:50 2022] GET /identity/v3/role_assignments?user.id=4a953762447e42cfa82713049dc178a3&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:51.045402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:51.110108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5494500-5301-4152-bc18-67f4ed639922 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:51.162276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947df7bb-4f85-41a1-be3f-bb1577bcfecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:51.163755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df7bb-4f85-41a1-be3f-bb1577bcfecc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.164247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df7bb-4f85-41a1-be3f-bb1577bcfecc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.164732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df7bb-4f85-41a1-be3f-bb1577bcfecc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.166277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947df7bb-4f85-41a1-be3f-bb1577bcfecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:51.167997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947df7bb-4f85-41a1-be3f-bb1577bcfecc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:51.171086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 78/148] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:51.176750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:51.256784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:51.323548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db640b0e-18fe-45a6-87a4-2964fbbf3520 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:51.375732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8588df-e7c6-4406-a37c-0337682b52b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:51.376440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8588df-e7c6-4406-a37c-0337682b52b9 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.376722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8588df-e7c6-4406-a37c-0337682b52b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.376998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8588df-e7c6-4406-a37c-0337682b52b9 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.377730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8588df-e7c6-4406-a37c-0337682b52b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:51.379267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8588df-e7c6-4406-a37c-0337682b52b9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:51.410835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 71/149] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:24:51 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:51.430592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0c7f1-6fd1-4c79-8723-16b92477b023 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.431223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0c7f1-6fd1-4c79-8723-16b92477b023 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.431700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0c7f1-6fd1-4c79-8723-16b92477b023 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.433302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 79/150] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:51.443655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16aba6-fced-45fe-b94c-2a1e8caf3653 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.444644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16aba6-fced-45fe-b94c-2a1e8caf3653 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.445092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16aba6-fced-45fe-b94c-2a1e8caf3653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.464591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8e16aba6-fced-45fe-b94c-2a1e8caf3653 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:51.466215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8e16aba6-fced-45fe-b94c-2a1e8caf3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:51.555166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 72/151] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:51.579555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0c7f1-6fd1-4c79-8723-16b92477b023 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:51.739334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0c7f1-6fd1-4c79-8723-16b92477b023 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:51.741024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064c97d-f3e8-4322-8a6f-aed53555d10e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.741537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064c97d-f3e8-4322-8a6f-aed53555d10e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.742065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064c97d-f3e8-4322-8a6f-aed53555d10e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.743475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 73/152] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:51.748466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210097fa-8665-4e68-86c6-bc5e0ddb2a09 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.748797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210097fa-8665-4e68-86c6-bc5e0ddb2a09 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.749088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210097fa-8665-4e68-86c6-bc5e0ddb2a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.756203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:51.771588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-210097fa-8665-4e68-86c6-bc5e0ddb2a09 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:51.771898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-210097fa-8665-4e68-86c6-bc5e0ddb2a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:51.788808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:51.789616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.789934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.790773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.806107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:51.809866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:51.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:51.821509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:51.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:51.824721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 80/153] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:51.831880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:51.863472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 74/154] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:24:51.893119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210097fa-8665-4e68-86c6-bc5e0ddb2a09 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:51.945644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deff786-8a3d-4346-b176-e97489d0a158 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:51.989865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:51.991174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:51.991616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:51.992047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:51.992980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:51.993438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:51.995237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:52.005614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 81/155] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:52.011539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:52.012879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210097fa-8665-4e68-86c6-bc5e0ddb2a09 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:52.054303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:52.059642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:52.060472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.060757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.061042 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.073365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:52.085689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:52.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:24:52.088201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 75/156] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:24:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:24:52.092975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:52.125162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:52.134489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619f5c2-9b48-40d0-8c36-b937f6bb86ad None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:52.178288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:52.179205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.179494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.179776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.181118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:52.182644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:52.184806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 82/157] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:52 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:52.190835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:52.214249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20be18f-5ae0-4db4-8f49-2c7bd23aaaca None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:52.255468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840cb9d-300c-431b-b50b-135d1a1668e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:52.256974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840cb9d-300c-431b-b50b-135d1a1668e7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.257266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840cb9d-300c-431b-b50b-135d1a1668e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.257549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840cb9d-300c-431b-b50b-135d1a1668e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.258365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840cb9d-300c-431b-b50b-135d1a1668e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:52.258679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840cb9d-300c-431b-b50b-135d1a1668e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:52.260093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840cb9d-300c-431b-b50b-135d1a1668e7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:52.272111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 76/158] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:24:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:52.277901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfedfa-9248-4a22-b02d-2b083b0c9386 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.278207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfedfa-9248-4a22-b02d-2b083b0c9386 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.278490 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfedfa-9248-4a22-b02d-2b083b0c9386 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.280063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 77/159] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:52.280648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfedfa-9248-4a22-b02d-2b083b0c9386 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:52.346876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f742a-7cac-456c-92f1-8455c49399bb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:52.350550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:52.393594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:52.394938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.395431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.395868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.408648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:52.429316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfedfa-9248-4a22-b02d-2b083b0c9386 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:52.482221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed536-a9f0-4a55-8ff0-22c282512459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:52.483825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed536-a9f0-4a55-8ff0-22c282512459 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.484199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed536-a9f0-4a55-8ff0-22c282512459 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.484609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed536-a9f0-4a55-8ff0-22c282512459 None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.485962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed536-a9f0-4a55-8ff0-22c282512459 None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:52.487509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed536-a9f0-4a55-8ff0-22c282512459 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:52.490586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:52.490866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 78/160] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:52 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:52.492481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:52.492972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f252ae-2e28-4d7b-be9d-b7b4732514a8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.493622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f252ae-2e28-4d7b-be9d-b7b4732514a8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.494088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f252ae-2e28-4d7b-be9d-b7b4732514a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.515955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-87f252ae-2e28-4d7b-be9d-b7b4732514a8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:52.517244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-87f252ae-2e28-4d7b-be9d-b7b4732514a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:52.601984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 79/161] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:52.603325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f252ae-2e28-4d7b-be9d-b7b4732514a8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:52.627929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 83/162] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:24:52.633923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:52.639713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:52.736501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:52.758806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f252ae-2e28-4d7b-be9d-b7b4732514a8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:52.851617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d51ccf-4ac1-4706-9b14-11b7df171eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:52.852364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d51ccf-4ac1-4706-9b14-11b7df171eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.852589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d51ccf-4ac1-4706-9b14-11b7df171eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:52.852869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d51ccf-4ac1-4706-9b14-11b7df171eb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:52.880735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b3df3-b0df-4c94-91fe-b14a8ded8fa3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:52.962184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d51ccf-4ac1-4706-9b14-11b7df171eb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:52.963725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d51ccf-4ac1-4706-9b14-11b7df171eb5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:52.998599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:52.999485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:52.999767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.000040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.006630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:53.008123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:53.009948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 84/163] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:52 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:53.010730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:53.090234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:53.125940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:53.126104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e373c-89f9-4f55-8558-4cab3433f924 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:53.132777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 80/164] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:53.137454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58424967-1ba7-4739-b007-d0351c894da0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:53.137454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58424967-1ba7-4739-b007-d0351c894da0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.137454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58424967-1ba7-4739-b007-d0351c894da0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.138349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 81/165] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:53.140761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58424967-1ba7-4739-b007-d0351c894da0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:53.162571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:53.163625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:53.163907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.164183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.181538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:53.191260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:53.192130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:53.193123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:53.195656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 85/166] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:53.196832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:53.318844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec841a0-7e86-40d7-bb7e-920d03424e79 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:53.355870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:53.362366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49006dd-9011-4fcb-923a-c337484aeea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:53.363508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49006dd-9011-4fcb-923a-c337484aeea9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:53.363993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49006dd-9011-4fcb-923a-c337484aeea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.364467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49006dd-9011-4fcb-923a-c337484aeea9 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.365955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49006dd-9011-4fcb-923a-c337484aeea9 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:53.367764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49006dd-9011-4fcb-923a-c337484aeea9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:53.369782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 86/167] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:53 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:53.371627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14eca6-71f2-45ac-8fa3-b7708bfd1e6b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:53.372124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14eca6-71f2-45ac-8fa3-b7708bfd1e6b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.372628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14eca6-71f2-45ac-8fa3-b7708bfd1e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.392247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5e14eca6-71f2-45ac-8fa3-b7708bfd1e6b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:53.393751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5e14eca6-71f2-45ac-8fa3-b7708bfd1e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:53.395427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58424967-1ba7-4739-b007-d0351c894da0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:53.470991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 87/168] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:53.472245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14eca6-71f2-45ac-8fa3-b7708bfd1e6b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:53.547113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:53.547790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:53.548081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.548364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.549323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=4466175761d84340a500132eb2b738d2) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:53.549520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:53.551019 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:53.571947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 82/169] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:53 2022] GET /identity/v3/role_assignments?user.id=4466175761d84340a500132eb2b738d2&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=9ebaaf52f4cd402880b403c50600168e => generated 608 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:53.573054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:53.623230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14eca6-71f2-45ac-8fa3-b7708bfd1e6b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:53.675088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:53.676558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:53.677148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.677638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.678677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:53.679380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:53.681033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:53.698298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:53.704081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 88/170] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:53.705074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:53.803722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:53.929948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca99428-ec74-4be6-a9d4-b333ec28c92d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:53.964613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02cec6-fec6-49f9-b883-3359dc2fca30 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:53.975471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:53.976886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:53.977169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:53.977443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:53.994843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:54.004011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:54.004869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:54.005666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 89/171] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:54.006381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:54.047557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:54.048225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.048507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.048781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.049614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] RBAC: Adding query filter params (role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=4466175761d84340a500132eb2b738d2) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:54.049949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:54.051396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:54.068924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 83/172] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:53 2022] GET /identity/v3/role_assignments?user.id=4466175761d84340a500132eb2b738d2&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 608 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:54.069931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:54.094539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:54.188592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73395b76-2694-4d5d-8ea3-6b9944711596 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:54.229643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:54.230990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.231501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.231979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] PATH_INFO: `/v3/users/4466175761d84340a500132eb2b738d2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.234100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] RBAC: Authorizing `identity:get_user(user_id=4466175761d84340a500132eb2b738d2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:54.235833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:54.237756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 90/173] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:54 2022] GET /identity/v3/users/4466175761d84340a500132eb2b738d2 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:54.247980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:54.262119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a75d-74f7-4039-891b-e5f05bbd9a5e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:54.292123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:54.345329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:54.346062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.346351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.346627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.349369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:54.349702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:54.351181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:54.366661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 84/174] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:54.377593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:54.475160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549b3f-60dc-4f01-8638-9163b4492460 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:54.503427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965da61c-85a0-4d23-a9e4-a6d16dfcd954 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:54.522000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:54.552983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9878a9d-b34e-4803-bc06-3e393abe6eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:54.553915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9878a9d-b34e-4803-bc06-3e393abe6eff None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.554345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9878a9d-b34e-4803-bc06-3e393abe6eff None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.555197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9878a9d-b34e-4803-bc06-3e393abe6eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.557649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:54.560995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.561464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.561938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] PATH_INFO: `/v3/users/2d7eac9126ed4f3d9c2cc54527bcaa3a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.570222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] RBAC: Authorizing `identity:get_user(user_id=2d7eac9126ed4f3d9c2cc54527bcaa3a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:54.571975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:54.573864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 85/175] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:54 2022] GET /identity/v3/users/2d7eac9126ed4f3d9c2cc54527bcaa3a => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:54.579355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:54.665137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9878a9d-b34e-4803-bc06-3e393abe6eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:54.666903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9878a9d-b34e-4803-bc06-3e393abe6eff None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:54.699077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:54.784013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012fbd-6dce-47eb-8d75-2bb5029f59c6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:54.796104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 91/176] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:54.797348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03aee3-8e86-4ab9-99c4-7dad738b598a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.802287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03aee3-8e86-4ab9-99c4-7dad738b598a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.802689 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03aee3-8e86-4ab9-99c4-7dad738b598a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.804276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 92/177] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:54.805072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03aee3-8e86-4ab9-99c4-7dad738b598a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:54.823838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157ae42-17d9-463a-8301-a782563677d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:54.824995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ae42-17d9-463a-8301-a782563677d4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.825280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ae42-17d9-463a-8301-a782563677d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.825562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157ae42-17d9-463a-8301-a782563677d4 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.827329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157ae42-17d9-463a-8301-a782563677d4 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:54.829695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157ae42-17d9-463a-8301-a782563677d4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:54.829830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:54.831427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 86/178] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:54 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:54.833393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8005-af9a-4af2-a980-2f40b2640838 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.833686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8005-af9a-4af2-a980-2f40b2640838 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.833985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e8005-af9a-4af2-a980-2f40b2640838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.857233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8b0e8005-af9a-4af2-a980-2f40b2640838 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:54.858837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8b0e8005-af9a-4af2-a980-2f40b2640838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:54.927478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03aee3-8e86-4ab9-99c4-7dad738b598a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:54.951155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 87/179] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:54.953383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e8005-af9a-4af2-a980-2f40b2640838 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:54.980585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:54.981352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:54.981606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:54.981889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] PATH_INFO: `/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:54.983397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] RBAC: Authorizing `identity:get_project(project_id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:54.985010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:54.986915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 93/180] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:54 2022] GET /identity/v3/projects/7dfb7fa2fcb84e4b8ed1e0aa0e431b1e => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:54.987789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.010307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:55.086932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e8005-af9a-4af2-a980-2f40b2640838 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:55.094723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:55.108192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49d0c2-c3ee-4388-9cc9-a2604b07b131 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:55.126741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:55.127640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.127942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.128259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/2d7eac9126ed4f3d9c2cc54527bcaa3a/roles/21c858abff6f4b9c90786fbcec903396` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.130292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=577d26cace0e4d3fba244021e34a6d6b, user_id=2d7eac9126ed4f3d9c2cc54527bcaa3a, role_id=21c858abff6f4b9c90786fbcec903396)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:55.131757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:55.145424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710e101-2632-4a54-95f9-d011bc3c377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:55.154085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 88/181] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:24:54 2022] PUT /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b/users/2d7eac9126ed4f3d9c2cc54527bcaa3a/roles/21c858abff6f4b9c90786fbcec903396 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:24:55.154311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710e101-2632-4a54-95f9-d011bc3c377d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.154311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710e101-2632-4a54-95f9-d011bc3c377d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.154311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710e101-2632-4a54-95f9-d011bc3c377d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.154543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.167485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710e101-2632-4a54-95f9-d011bc3c377d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f710e101-2632-4a54-95f9-d011bc3c377d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:55.175625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:55.176576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:55.177346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 94/182] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:55.178501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7de6-2ba6-4ad7-8b9e-095f27cc2879 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.178604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7de6-2ba6-4ad7-8b9e-095f27cc2879 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.178867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab7de6-2ba6-4ad7-8b9e-095f27cc2879 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.181197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 95/183] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:55.185897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d9dc5-8e63-4766-838f-0cc8da0877f1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.186832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d9dc5-8e63-4766-838f-0cc8da0877f1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.187223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d9dc5-8e63-4766-838f-0cc8da0877f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.205204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b23d9dc5-8e63-4766-838f-0cc8da0877f1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:55.206610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b23d9dc5-8e63-4766-838f-0cc8da0877f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:55.296199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d11701-6fc6-48d2-9c35-3576f77f38e9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:55.296656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 96/184] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:55.297728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d9dc5-8e63-4766-838f-0cc8da0877f1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.303961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:55.378073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f120aa5-9992-4910-8895-0c11be9817cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:55.378818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120aa5-9992-4910-8895-0c11be9817cf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.379357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120aa5-9992-4910-8895-0c11be9817cf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.379638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120aa5-9992-4910-8895-0c11be9817cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.380894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f120aa5-9992-4910-8895-0c11be9817cf None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e, user.id=4466175761d84340a500132eb2b738d2) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:55.381214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f120aa5-9992-4910-8895-0c11be9817cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:55.382684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f120aa5-9992-4910-8895-0c11be9817cf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:55.391278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 89/185] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:55 2022] GET /identity/v3/role_assignments?user.id=4466175761d84340a500132eb2b738d2&scope.project.id=7dfb7fa2fcb84e4b8ed1e0aa0e431b1e&role.id=9ebaaf52f4cd402880b403c50600168e => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:55.397890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3ecc9-770e-4c65-b06a-e61f11bf3006 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.397890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3ecc9-770e-4c65-b06a-e61f11bf3006 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.397890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3ecc9-770e-4c65-b06a-e61f11bf3006 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.397890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 90/186] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:55.397890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3ecc9-770e-4c65-b06a-e61f11bf3006 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.436590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:55.463302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d9dc5-8e63-4766-838f-0cc8da0877f1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:55.509334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c2ed2-a3eb-4a72-b7f0-b0bfa46e3daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:55.510050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2ed2-a3eb-4a72-b7f0-b0bfa46e3daf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.510332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2ed2-a3eb-4a72-b7f0-b0bfa46e3daf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.510706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2ed2-a3eb-4a72-b7f0-b0bfa46e3daf None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.511471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c2ed2-a3eb-4a72-b7f0-b0bfa46e3daf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:55.511778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c2ed2-a3eb-4a72-b7f0-b0bfa46e3daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:55.513367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c2ed2-a3eb-4a72-b7f0-b0bfa46e3daf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:55.529864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 97/187] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:55.531698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6897a-5567-4075-b8a0-057b8186e760 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.532096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6897a-5567-4075-b8a0-057b8186e760 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.533750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6897a-5567-4075-b8a0-057b8186e760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.543760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3ecc9-770e-4c65-b06a-e61f11bf3006 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:55.553869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-72d6897a-5567-4075-b8a0-057b8186e760 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:55.554056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-72d6897a-5567-4075-b8a0-057b8186e760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:55.619255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:55.620837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.621325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.621790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.624324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:55.625896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:55.627827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 91/188] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:55.628616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.661886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 98/189] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:55.661886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6897a-5567-4075-b8a0-057b8186e760 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.691526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:55.770559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:55.790532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fbcb6-2c95-447b-bf6c-0dc2be263259 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:55.814121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6897a-5567-4075-b8a0-057b8186e760 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:55.835984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:55.836857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.837139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.837412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] PATH_INFO: `/v3/users/ef205a80569946beaa49a091ba7df4ad` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.854755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] RBAC: Authorizing `identity:get_user(user_id=ef205a80569946beaa49a091ba7df4ad)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:55.856684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:55.858461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 92/190] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:55 2022] GET /identity/v3/users/ef205a80569946beaa49a091ba7df4ad => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:55.859225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.865309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:55.866294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:55.866673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:55.867014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:55.867818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:55.869315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:55.897892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 99/191] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:24:55 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:55.898657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:55.976459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:55.988592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3e29c-0dfe-4015-b2fc-06a2676a57f0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:56.029318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:56.030556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:56.030556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:56.030832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:56.043357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:56.050355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef751bc-4a26-4474-8844-ceeaa732beb0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:56.054034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:56.055214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:24:56.056811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 93/192] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:24:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:56.060644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:56.093736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:56.105485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:56.106384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:56.106668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:56.106943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:56.182171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc2014-5346-4c48-a724-d66ce932a654 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:56.193677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:56.196807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:56.226036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:56.226815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:56.227400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:56.227504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:56.228286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:56.228668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:56.230187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:56.240559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 94/193] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:24:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:56.246620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:56.268324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:56.317989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 100/194] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:56.323964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:56.360727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:56.366978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704ef3d-a0a1-4566-8d18-b3419e903a36 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:56.408787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:56.409784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:56.410086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:56.410360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] PATH_INFO: `/v3/users/ef205a80569946beaa49a091ba7df4ad` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:56.411645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] RBAC: Authorizing `identity:get_user(user_id=ef205a80569946beaa49a091ba7df4ad)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:56.413464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:56.415235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 95/195] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:56 2022] GET /identity/v3/users/ef205a80569946beaa49a091ba7df4ad => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:56.421781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:56.470724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60517cd9-5b7b-45aa-98e1-ba980aa44519 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:56.521658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:56.523035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:56.523314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:56.523588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:56.526421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:56.527857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:56.530277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 101/196] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:56 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:56.535151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:56.537065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:56.562865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263983-ea46-41bf-ad58-f0237d3805ec None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:56.662574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:56.663511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:56.663801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:56.664081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:56.678179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:56.687440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6d536-6d16-49cf-b49d-688fd24215ab None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:56.753281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:56.753989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:56.754273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:56.754605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:56.755503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=ef205a80569946beaa49a091ba7df4ad) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:56.755824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:56.757259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:56.762893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:56.765040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:56.770638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 102/197] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:56 2022] GET /identity/v3/role_assignments?user.id=ef205a80569946beaa49a091ba7df4ad&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:56.855859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:56.882103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 96/198] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:56.883584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:57.007239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e377cd1-6c92-4c71-b5da-3e49c10505c5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:57.050097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677663a-bb24-4667-b815-efc1ca0a71e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:57.051246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677663a-bb24-4667-b815-efc1ca0a71e2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:57.051668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677663a-bb24-4667-b815-efc1ca0a71e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:57.052087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677663a-bb24-4667-b815-efc1ca0a71e2 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:57.054652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677663a-bb24-4667-b815-efc1ca0a71e2 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:57.056143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677663a-bb24-4667-b815-efc1ca0a71e2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:57.057969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 103/199] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:56 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:57.066202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:57.205380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c90a-a47f-4efb-acb9-a066058da25a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:57.267226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd03138-49da-4535-adee-850fc7fb23f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:57.267840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd03138-49da-4535-adee-850fc7fb23f1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:57.268131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd03138-49da-4535-adee-850fc7fb23f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:57.268413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd03138-49da-4535-adee-850fc7fb23f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:57.269264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd03138-49da-4535-adee-850fc7fb23f1 None admin] RBAC: Adding query filter params (role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=ef205a80569946beaa49a091ba7df4ad) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:57.269580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd03138-49da-4535-adee-850fc7fb23f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:57.271067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd03138-49da-4535-adee-850fc7fb23f1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:57.289772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 97/200] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:57 2022] GET /identity/v3/role_assignments?user.id=ef205a80569946beaa49a091ba7df4ad&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:57.541911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead4a7-c03e-4cda-9818-2335e84b066c None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:57.542205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead4a7-c03e-4cda-9818-2335e84b066c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:57.542400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ead4a7-c03e-4cda-9818-2335e84b066c None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:57.544087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 104/201] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:57.550722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10502ab-12f8-41a4-a729-5f61937b54e6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:57.550722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10502ab-12f8-41a4-a729-5f61937b54e6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:57.551289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10502ab-12f8-41a4-a729-5f61937b54e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:57.569112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a10502ab-12f8-41a4-a729-5f61937b54e6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:57.570345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a10502ab-12f8-41a4-a729-5f61937b54e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:57.617705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:57.654612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 98/202] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:57.678390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ead4a7-c03e-4cda-9818-2335e84b066c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:57.714700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:57.816539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ead4a7-c03e-4cda-9818-2335e84b066c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:57.868394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:57.869500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:57.869985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:57.870459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:57.890684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:57.899054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:57.899990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:24:57.900776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:57.903109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 105/203] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:57.908936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10502ab-12f8-41a4-a729-5f61937b54e6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:58.013137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10502ab-12f8-41a4-a729-5f61937b54e6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:58.048895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:58.049800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:58.050271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:58.050699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:58.051589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:58.052035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:58.053554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:58.062246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 99/204] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:58.067954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:58.173873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:58.194061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12637979-60d4-4bb3-a242-b032960f8389 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:58.244418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:58.245548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:58.246024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:58.246504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] PATH_INFO: `/v3/users/2d7eac9126ed4f3d9c2cc54527bcaa3a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:58.247920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] RBAC: Authorizing `identity:get_user(user_id=2d7eac9126ed4f3d9c2cc54527bcaa3a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:58.249665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:58.251537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 106/205] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:58 2022] GET /identity/v3/users/2d7eac9126ed4f3d9c2cc54527bcaa3a => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:58.257089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:58.305059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:58.369739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e5c36-45ab-478e-87fe-e8d5737b4df1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:58.405921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840ed66-b3a0-46a1-ac0f-53837d03879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:58.406866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ed66-b3a0-46a1-ac0f-53837d03879c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:58.407292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ed66-b3a0-46a1-ac0f-53837d03879c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:58.407731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ed66-b3a0-46a1-ac0f-53837d03879c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:58.478970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840ed66-b3a0-46a1-ac0f-53837d03879c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:58.480671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840ed66-b3a0-46a1-ac0f-53837d03879c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:58.520408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:58.582986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 100/206] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:24:58.588989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:58.743893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965186-4183-4f87-9808-bfee59f38a4e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:58.756199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8233e-b9e8-4594-b908-2cfce1e5ba7e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:58.756668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8233e-b9e8-4594-b908-2cfce1e5ba7e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:58.757085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8233e-b9e8-4594-b908-2cfce1e5ba7e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:58.758650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 101/207] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:24:58.763457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f284-cc65-47b8-8782-119683ff6f67 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:58.764421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f284-cc65-47b8-8782-119683ff6f67 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:58.764838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f284-cc65-47b8-8782-119683ff6f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:58.787159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ab80f284-cc65-47b8-8782-119683ff6f67 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:58.790173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ab80f284-cc65-47b8-8782-119683ff6f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:58.825785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:58.827476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:58.827900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:58.828315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:58.831870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:58.833470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:58.836724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 107/208] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:24:58 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:58.854143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:58.895822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:58.922199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 102/209] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:24:58.971743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80f284-cc65-47b8-8782-119683ff6f67 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:59.049259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263de6a0-55d9-4f57-9daa-bb19d043f485 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:59.062701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:59.098503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c8c31-f72d-4970-9dc0-4827bb0c518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:59.099650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c8c31-f72d-4970-9dc0-4827bb0c518e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.100353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c8c31-f72d-4970-9dc0-4827bb0c518e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.100774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c8c31-f72d-4970-9dc0-4827bb0c518e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.102459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c8c31-f72d-4970-9dc0-4827bb0c518e None admin] RBAC: Adding query filter params (role.id=21c858abff6f4b9c90786fbcec903396, scope.project.id=577d26cace0e4d3fba244021e34a6d6b, user.id=2d7eac9126ed4f3d9c2cc54527bcaa3a) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:59.102923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c8c31-f72d-4970-9dc0-4827bb0c518e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:59.104315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80f284-cc65-47b8-8782-119683ff6f67 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:59.104502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c8c31-f72d-4970-9dc0-4827bb0c518e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:59.112274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 108/210] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:24:58 2022] GET /identity/v3/role_assignments?user.id=2d7eac9126ed4f3d9c2cc54527bcaa3a&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=21c858abff6f4b9c90786fbcec903396 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:59.117866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9daca3-e402-4f35-8ff0-61aceb419434 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.118698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9daca3-e402-4f35-8ff0-61aceb419434 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.118698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9daca3-e402-4f35-8ff0-61aceb419434 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.122171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 109/211] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:59.130794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8d70c-2947-439d-ab04-b53473a97b15 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.131101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8d70c-2947-439d-ab04-b53473a97b15 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.131344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8d70c-2947-439d-ab04-b53473a97b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.165070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bcb8d70c-2947-439d-ab04-b53473a97b15 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:59.166226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bcb8d70c-2947-439d-ab04-b53473a97b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:59.189253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:59.190096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.190426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.190697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.191882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:59.193364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:59.196643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 103/212] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:24:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:59.202360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:59.313690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 110/213] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:59.339108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8d70c-2947-439d-ab04-b53473a97b15 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:59.364876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:59.369504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a78b9-5231-4594-8e80-db68e48c8676 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:59.417360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:59.418024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.418308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.418582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.419652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:59.421958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:59.431533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:59.454077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 104/214] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:24:59 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:24:59.484451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8d70c-2947-439d-ab04-b53473a97b15 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:59.527305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae177b6-5c15-4d50-84d7-05fc467cc0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:59.528616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae177b6-5c15-4d50-84d7-05fc467cc0ac None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.529121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae177b6-5c15-4d50-84d7-05fc467cc0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.530227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae177b6-5c15-4d50-84d7-05fc467cc0ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.544242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae177b6-5c15-4d50-84d7-05fc467cc0ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9ae177b6-5c15-4d50-84d7-05fc467cc0ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:59.553504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:59.554477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:24:59.555252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:59.562346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 111/215] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:24:59.564629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:59.590693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097edb-f29e-4ae0-845e-bbae0534a480 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.591309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097edb-f29e-4ae0-845e-bbae0534a480 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.591779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2097edb-f29e-4ae0-845e-bbae0534a480 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.594706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 112/216] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:24:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:24:59.601670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fe95a-bd29-4cd0-b8c9-1e2dcf6faeba None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.602762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fe95a-bd29-4cd0-b8c9-1e2dcf6faeba None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.603194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fe95a-bd29-4cd0-b8c9-1e2dcf6faeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.640343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6e0fe95a-bd29-4cd0-b8c9-1e2dcf6faeba None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:24:59.641572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6e0fe95a-bd29-4cd0-b8c9-1e2dcf6faeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:24:59.715373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e466fb-aa26-4342-b744-66af70cf5a00 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:59.736181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 113/217] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:24:59.756066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:59.756807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.757288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.757406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.758148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:24:59.758260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fe95a-bd29-4cd0-b8c9-1e2dcf6faeba None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:59.758452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:59.759906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:59.768213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:59.768741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 105/218] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:24:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:24:59.774422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:59.847573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:24:59.892548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fe95a-bd29-4cd0-b8c9-1e2dcf6faeba None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:59.905573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215aba8a-f76e-400a-8f01-d43d2c960eae None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:24:59.934623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:59.935603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.936001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.936285 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.949926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:24:59.951059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:24:59.952334 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:24:59.952808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:24:59.953267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] PATH_INFO: `/v3/users/85f0126e7b1e4980835af0da6f34e1ee` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:24:59.960692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:24:59.961599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:24:59.962406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 114/219] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:24:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:24:59.970118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:24:59.974863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] RBAC: Authorizing `identity:get_user(user_id=85f0126e7b1e4980835af0da6f34e1ee)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:24:59.976616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:24:59.978626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 106/220] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:24:59 2022] GET /identity/v3/users/85f0126e7b1e4980835af0da6f34e1ee => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:24:59.985902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.089591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:00.133802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793b5a2-0267-453b-bfa5-f4e69c28e9ce None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:00.155285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1f01-7537-4a17-a1d1-b9cedbb514fa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:00.174910 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:00.178120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338d5cb-c4a0-47ef-a730-5143d2dae00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:00.179111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338d5cb-c4a0-47ef-a730-5143d2dae00c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.179602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338d5cb-c4a0-47ef-a730-5143d2dae00c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.180234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338d5cb-c4a0-47ef-a730-5143d2dae00c None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.181524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338d5cb-c4a0-47ef-a730-5143d2dae00c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:00.182084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338d5cb-c4a0-47ef-a730-5143d2dae00c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:00.184105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338d5cb-c4a0-47ef-a730-5143d2dae00c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:00.195962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 115/221] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:24:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:00.202334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7eec6-e071-46dc-82f8-6a7a13388ae3 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.202451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:00.202451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.202451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.202451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.203223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7eec6-e071-46dc-82f8-6a7a13388ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.203718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7eec6-e071-46dc-82f8-6a7a13388ae3 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.206328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 116/222] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:00.207340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7eec6-e071-46dc-82f8-6a7a13388ae3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.289433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:00.292333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:00.356876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7eec6-e071-46dc-82f8-6a7a13388ae3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:00.385349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:00.399065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b33745-a3db-4d0b-82c9-7e3ac73de3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:00.399954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b33745-a3db-4d0b-82c9-7e3ac73de3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.400241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b33745-a3db-4d0b-82c9-7e3ac73de3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.400519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b33745-a3db-4d0b-82c9-7e3ac73de3b3 None admin] PATH_INFO: `/v3/users/ef205a80569946beaa49a091ba7df4ad` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.402824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b33745-a3db-4d0b-82c9-7e3ac73de3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=ef205a80569946beaa49a091ba7df4ad)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:00.405438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b33745-a3db-4d0b-82c9-7e3ac73de3b3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:00.408081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 117/223] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:00 2022] GET /identity/v3/users/ef205a80569946beaa49a091ba7df4ad => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:00.409426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1119b8-9ec8-4d8d-bc44-5590a68dc8fd None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.409939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1119b8-9ec8-4d8d-bc44-5590a68dc8fd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.410393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1119b8-9ec8-4d8d-bc44-5590a68dc8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.419061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 107/224] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:00.419061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.432356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6d1119b8-9ec8-4d8d-bc44-5590a68dc8fd None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:00.433691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6d1119b8-9ec8-4d8d-bc44-5590a68dc8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:00.439788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:00.506934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 118/225] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:00.507714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1119b8-9ec8-4d8d-bc44-5590a68dc8fd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.556750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7a618-b3ff-4928-a818-1c5e19b350ca None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:00.594457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:00.595562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.595908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.596241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.597647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:00.599335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:00.601343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 108/226] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:00 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:00.606721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.643875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1119b8-9ec8-4d8d-bc44-5590a68dc8fd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:00.677874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:00.691394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:00.693206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.693705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.694384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.696416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:00.698697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:00.700811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 119/227] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:00 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:00.706170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.731636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:00.759893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a521b0-c8e2-4ea4-995a-88a4876541bf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:00.807286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:00.808402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.808859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.809302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.823346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671607-9bb9-485f-8693-84f1cd45266f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:00.824266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:00.835158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:00.836039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:00.836818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:00.839247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 109/228] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:00.840489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.870945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:00.872739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:00.873223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:00.874446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/ef205a80569946beaa49a091ba7df4ad/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:00.877508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3bce33fec47b5b23c70bedccc7dc9, user_id=ef205a80569946beaa49a091ba7df4ad, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:00.879814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:00.896529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 120/229] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:00 2022] PUT /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/ef205a80569946beaa49a091ba7df4ad/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:00.897680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:00.981413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72850137-e34a-4c0d-b5f3-d5196d737697 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:01.022733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:01.023660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:01.024120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:01.024548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:01.025540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=85f0126e7b1e4980835af0da6f34e1ee) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:01.026049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:01.027668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:01.028828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:01.037268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 110/230] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:00 2022] GET /identity/v3/role_assignments?user.id=85f0126e7b1e4980835af0da6f34e1ee&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=9ebaaf52f4cd402880b403c50600168e => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:01.120860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bb5d-b4e8-42d6-a228-d284c240f7da None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:01.161979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:01.163574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:01.164082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:01.164464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:01.165241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:01.165551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:01.167029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:01.178410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 121/231] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:01.184245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:01.189829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:01.296151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed2af1-69b4-48e8-bfdb-34b25dbd5b34 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:01.330734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:01.331613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:01.331891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:01.332159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] PATH_INFO: `/v3/users/ef205a80569946beaa49a091ba7df4ad` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:01.333349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] RBAC: Authorizing `identity:get_user(user_id=ef205a80569946beaa49a091ba7df4ad)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:01.334824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:01.336454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 111/232] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:01 2022] GET /identity/v3/users/ef205a80569946beaa49a091ba7df4ad => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:01.342735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:01.476813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284f42-f360-4007-9de2-c08efac82ed8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:01.500086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:01.519815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafac9ee-8d7d-478e-8a5b-57b70418de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:01.520682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafac9ee-8d7d-478e-8a5b-57b70418de4e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:01.520959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafac9ee-8d7d-478e-8a5b-57b70418de4e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:01.521227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafac9ee-8d7d-478e-8a5b-57b70418de4e None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:01.522436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafac9ee-8d7d-478e-8a5b-57b70418de4e None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:01.523865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafac9ee-8d7d-478e-8a5b-57b70418de4e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:01.525568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 122/233] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:01 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:01.531446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:01.605760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:01.660004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3dd1c-5b49-4a3f-a275-441dc8206ccb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:01.716889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89128fdf-3e10-406e-992a-ad41efd45e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:01.718206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128fdf-3e10-406e-992a-ad41efd45e25 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:01.718689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128fdf-3e10-406e-992a-ad41efd45e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:01.719156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89128fdf-3e10-406e-992a-ad41efd45e25 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/ef205a80569946beaa49a091ba7df4ad/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:01.723955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89128fdf-3e10-406e-992a-ad41efd45e25 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3bce33fec47b5b23c70bedccc7dc9, user_id=ef205a80569946beaa49a091ba7df4ad, role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:01.725633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89128fdf-3e10-406e-992a-ad41efd45e25 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:01.745103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 112/234] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:01 2022] PUT /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/ef205a80569946beaa49a091ba7df4ad/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:02.062720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a221-b6d8-49af-94b1-69a0e5aead42 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:02.063195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a221-b6d8-49af-94b1-69a0e5aead42 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:02.063195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275a221-b6d8-49af-94b1-69a0e5aead42 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:02.064588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 123/235] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:02.069959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38591c90-6b26-432c-ac44-b67458e90919 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:02.070759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38591c90-6b26-432c-ac44-b67458e90919 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:02.071239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38591c90-6b26-432c-ac44-b67458e90919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:02.092846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-38591c90-6b26-432c-ac44-b67458e90919 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:02.094391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-38591c90-6b26-432c-ac44-b67458e90919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:02.175304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 113/236] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:02.212589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275a221-b6d8-49af-94b1-69a0e5aead42 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:02.371043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275a221-b6d8-49af-94b1-69a0e5aead42 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:02.375337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8e74e-7f6d-4220-b2c1-63646d3e25da None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:02.375928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8e74e-7f6d-4220-b2c1-63646d3e25da None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:02.376429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8e74e-7f6d-4220-b2c1-63646d3e25da None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:02.378302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 114/237] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:02.408434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb18f71-754e-4b36-9ef0-aabae6957468 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:02.408715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb18f71-754e-4b36-9ef0-aabae6957468 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:02.409154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb18f71-754e-4b36-9ef0-aabae6957468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:02.414743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:02.415640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:02.415933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:02.416213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:02.417869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:02.418997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:02.421278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 124/238] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:02.427699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:02.443077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4bb18f71-754e-4b36-9ef0-aabae6957468 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:02.444570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4bb18f71-754e-4b36-9ef0-aabae6957468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:02.510353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:02.519329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:02.533888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 115/239] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:02.579696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb18f71-754e-4b36-9ef0-aabae6957468 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:02.608069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907fad-1193-4734-8b37-c3f5709addcc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:02.683121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:02.683808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:02.684091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:02.684363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:02.685071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:02.687064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:02.725607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 125/240] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:02 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:25:02.776243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb18f71-754e-4b36-9ef0-aabae6957468 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:02.829701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:02.830962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:02.831527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:02.831999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:02.848049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:02.860589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:02.861540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:02.862366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:02.864880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 116/241] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:02.871125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:02.997711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17596df-98cc-46ad-b6d2-e69c5be848ea None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:03.007687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:03.034366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:03.035044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.035334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.035608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.036346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:03.036653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:03.038303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:03.053586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 126/242] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:03.058931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:03.122847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:03.182652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81849a4e-083d-4a1c-8ba6-9fe86eef84a0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:03.222701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:03.223601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.223888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.224165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] PATH_INFO: `/v3/users/d0dca047615643dabd79fa05ef786402` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.242272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] RBAC: Authorizing `identity:get_user(user_id=d0dca047615643dabd79fa05ef786402)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:03.243793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:03.247386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 117/243] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:03 2022] GET /identity/v3/users/d0dca047615643dabd79fa05ef786402 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:03.252106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:03.377090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860ecc4-f7e0-4409-aaad-bf4534f8d92c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:03.418268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b656f5-0f23-45a2-9b54-a323637c3fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:03.419281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b656f5-0f23-45a2-9b54-a323637c3fda None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.419787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b656f5-0f23-45a2-9b54-a323637c3fda None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.420844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b656f5-0f23-45a2-9b54-a323637c3fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.462621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:03.494007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b656f5-0f23-45a2-9b54-a323637c3fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:03.495487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b656f5-0f23-45a2-9b54-a323637c3fda None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:03.604461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 127/244] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:03.611009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:03.618426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f5766-c351-49e9-b76a-0dd9029ff239 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.618717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f5766-c351-49e9-b76a-0dd9029ff239 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.618993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f5766-c351-49e9-b76a-0dd9029ff239 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.620389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 128/245] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:03.625579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd071-d051-4372-91e7-a0b040f63824 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.625888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd071-d051-4372-91e7-a0b040f63824 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.626176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd071-d051-4372-91e7-a0b040f63824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.659237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9c1fd071-d051-4372-91e7-a0b040f63824 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:03.661823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9c1fd071-d051-4372-91e7-a0b040f63824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:03.793016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c185c4-5318-4f58-ae28-4c913543d0cd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:03.811861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:03.812907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:03.820541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 129/246] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:03.821872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303aa28-d3de-4920-9c13-fe6653809d4d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.822165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303aa28-d3de-4920-9c13-fe6653809d4d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.822447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303aa28-d3de-4920-9c13-fe6653809d4d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.823791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 130/247] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:03.832809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ed366-b785-4b8f-bc9a-b48111dcb3fa None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.833095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ed366-b785-4b8f-bc9a-b48111dcb3fa None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.833375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ed366-b785-4b8f-bc9a-b48111dcb3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.867505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-767ed366-b785-4b8f-bc9a-b48111dcb3fa None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:03.868658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-767ed366-b785-4b8f-bc9a-b48111dcb3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:03.876372 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:03.877261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:03.877542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:03.877815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:03.879087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:03.880655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:03.882497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 118/248] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:03 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:03.883080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:03.995302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 131/249] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:03.996171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ed366-b785-4b8f-bc9a-b48111dcb3fa None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:04.050115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61178a-3c54-40b6-9f6b-9f6302f5bcf1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:04.088328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:04.089142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.089426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.089703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.101416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:04.110153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:04.111028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:04.111801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 119/250] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:04.112459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:04.176871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ed366-b785-4b8f-bc9a-b48111dcb3fa None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:04.181370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:04.229551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:04.241760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:04.242472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.242755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.243032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.243880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] RBAC: Adding query filter params (role.id=21c858abff6f4b9c90786fbcec903396, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=d0dca047615643dabd79fa05ef786402) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:04.244196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:04.245633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:04.257221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 132/251] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:03 2022] GET /identity/v3/role_assignments?user.id=d0dca047615643dabd79fa05ef786402&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=21c858abff6f4b9c90786fbcec903396 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:04.258088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:04.297906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c475990-b905-4927-b762-bc4126321fd5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:04.363980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:04.364807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.365089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.365360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.381497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:04.398282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:04.399222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:04.400007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:04.405512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 120/252] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:04.413030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:04.456152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d131b-0a06-474a-af2d-3977cd351d56 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:04.535908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d2ae2-51ce-4199-9a89-a64d509c9e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:04.536616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2ae2-51ce-4199-9a89-a64d509c9e67 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.536950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2ae2-51ce-4199-9a89-a64d509c9e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.537227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d2ae2-51ce-4199-9a89-a64d509c9e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.538283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d2ae2-51ce-4199-9a89-a64d509c9e67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:04.538816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d2ae2-51ce-4199-9a89-a64d509c9e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:04.540364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d2ae2-51ce-4199-9a89-a64d509c9e67 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:04.550029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 133/253] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:04.551704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb05ac2-d2b2-43df-a6a2-f948f71240fc None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.552190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb05ac2-d2b2-43df-a6a2-f948f71240fc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.552684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb05ac2-d2b2-43df-a6a2-f948f71240fc None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.554216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 134/254] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:04.555097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb05ac2-d2b2-43df-a6a2-f948f71240fc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:04.575062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:04.579429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269a974-4e54-4839-80ef-b4360c9caccc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:04.633635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58caeb1-e607-4ff6-bfe3-b047e3bb474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:04.634540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58caeb1-e607-4ff6-bfe3-b047e3bb474e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.634890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58caeb1-e607-4ff6-bfe3-b047e3bb474e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.635552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58caeb1-e607-4ff6-bfe3-b047e3bb474e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.636393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58caeb1-e607-4ff6-bfe3-b047e3bb474e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:04.636767 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58caeb1-e607-4ff6-bfe3-b047e3bb474e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:04.639719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58caeb1-e607-4ff6-bfe3-b047e3bb474e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:04.642190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:04.658863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 121/255] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:04.660328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd47eb-2ec7-4ed4-955b-e4c22fcaeeef None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.660741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd47eb-2ec7-4ed4-955b-e4c22fcaeeef None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.661113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd47eb-2ec7-4ed4-955b-e4c22fcaeeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.689114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb05ac2-d2b2-43df-a6a2-f948f71240fc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:04.700992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-20cd47eb-2ec7-4ed4-955b-e4c22fcaeeef None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:04.702791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-20cd47eb-2ec7-4ed4-955b-e4c22fcaeeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:04.753877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:04.754761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:04.755044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:04.755316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] PATH_INFO: `/v3/users/85f0126e7b1e4980835af0da6f34e1ee` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:04.757101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] RBAC: Authorizing `identity:get_user(user_id=85f0126e7b1e4980835af0da6f34e1ee)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:04.758589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:04.765877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 135/256] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:04 2022] GET /identity/v3/users/85f0126e7b1e4980835af0da6f34e1ee => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:04.766673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:04.836198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 122/257] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:04.836893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd47eb-2ec7-4ed4-955b-e4c22fcaeeef None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:04.996456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb969d-3706-4072-8ff8-ba23bbda89aa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:05.026616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:05.041210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:05.042520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:05.042999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:05.043638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] PATH_INFO: `/v3/users/ef205a80569946beaa49a091ba7df4ad` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:05.045079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=ef205a80569946beaa49a091ba7df4ad)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:05.046752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:05.048952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 136/258] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:04 2022] GET /identity/v3/users/ef205a80569946beaa49a091ba7df4ad => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:05.050105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:05.090214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:05.140897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd47eb-2ec7-4ed4-955b-e4c22fcaeeef None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:05.165798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0874fa-e578-413f-b762-b88120405cc0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:05.192753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:05.193575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:05.194040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:05.194513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:05.198057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:05.199513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:05.201959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 123/259] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:04 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:05.203581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:05.228932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:05.229662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:05.229969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:05.230259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:05.253532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:05.261591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:05.262874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 137/260] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:05.263691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:05.388225 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:05.403349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dac700-6579-47ab-a82d-0df62a8b35c6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:05.475379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:05.476052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:05.476337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:05.476611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:05.492698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895ffe7-221f-40a9-b40d-bb3fd21513eb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:05.558211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:05.563980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:05.563980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:05.563980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:05.563980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/85f0126e7b1e4980835af0da6f34e1ee/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:05.568735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3bce33fec47b5b23c70bedccc7dc9, user_id=85f0126e7b1e4980835af0da6f34e1ee, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:05.570243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:05.605658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 138/261] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:05 2022] PUT /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/85f0126e7b1e4980835af0da6f34e1ee/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:05.608657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:05.637009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:05.638640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:05.787652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 124/262] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:05.793195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:05.867498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:05.914419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc05874-0387-45df-b2e6-93ab5285a225 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:05.940139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5c5a-4c25-4636-ba74-1e32a302f15e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:05.952236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:05.953050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:05.953338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:05.953619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:05.957407 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:05.961552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:05.962279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 125/263] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:05 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:05.964831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:05.966392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:06.028858 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:06.029644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.029891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.030173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.030937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:06.031245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:06.032628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:06.048357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 139/264] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:06.053413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:06.089966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f73c1f-23cb-458f-a48e-74e970dde65d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:06.120960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:06.133620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccb8b3-0507-4ec8-a4fe-594ccd6e274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:06.135711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccb8b3-0507-4ec8-a4fe-594ccd6e274a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.136198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccb8b3-0507-4ec8-a4fe-594ccd6e274a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.136686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccb8b3-0507-4ec8-a4fe-594ccd6e274a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.137717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccb8b3-0507-4ec8-a4fe-594ccd6e274a None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=ef205a80569946beaa49a091ba7df4ad) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:06.138242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccb8b3-0507-4ec8-a4fe-594ccd6e274a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:06.139911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccb8b3-0507-4ec8-a4fe-594ccd6e274a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:06.147980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 126/265] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:05 2022] GET /identity/v3/role_assignments?user.id=ef205a80569946beaa49a091ba7df4ad&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:06.243259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caeccb5-51f3-4041-b287-6b66d7fbee13 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:06.290812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412073c-7daf-4a42-9bc6-6bbad5435d90 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.291774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412073c-7daf-4a42-9bc6-6bbad5435d90 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.292264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412073c-7daf-4a42-9bc6-6bbad5435d90 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.293918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 127/266] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:06.301225 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:06.302167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.302454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.302729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] PATH_INFO: `/v3/users/ef205a80569946beaa49a091ba7df4ad` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.303220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f0c38-88e6-419e-bc4d-a4c820d6c084 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.303800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f0c38-88e6-419e-bc4d-a4c820d6c084 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.304293 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f0c38-88e6-419e-bc4d-a4c820d6c084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.304931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] RBAC: Authorizing `identity:get_user(user_id=ef205a80569946beaa49a091ba7df4ad)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:06.306529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:06.308503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 140/267] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:06 2022] GET /identity/v3/users/ef205a80569946beaa49a091ba7df4ad => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:06.313925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:06.326678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cf5f0c38-88e6-419e-bc4d-a4c820d6c084 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:06.328825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cf5f0c38-88e6-419e-bc4d-a4c820d6c084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:06.373053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:06.424309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 128/268] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:06.442813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007734ba-eb01-4a0d-9c99-0a74d741272f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.443164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007734ba-eb01-4a0d-9c99-0a74d741272f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.443375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007734ba-eb01-4a0d-9c99-0a74d741272f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.444858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 129/269] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:06.445087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7da35c-fa16-4445-8c4c-9bcf1a924a0b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:06.450128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abb9f2-04e5-45c3-a1ea-2550dfa27fe0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.450467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abb9f2-04e5-45c3-a1ea-2550dfa27fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.450745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abb9f2-04e5-45c3-a1ea-2550dfa27fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.470094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c9abb9f2-04e5-45c3-a1ea-2550dfa27fe0 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:06.471447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c9abb9f2-04e5-45c3-a1ea-2550dfa27fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:06.497793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:06.498633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.498926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.499261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.557069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 130/270] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:06.558404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abb9f2-04e5-45c3-a1ea-2550dfa27fe0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:06.581461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:06.583002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:06.621160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:06.641869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:06.692137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abb9f2-04e5-45c3-a1ea-2550dfa27fe0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:06.715254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 141/271] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:06.716407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:06.735921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:06.737040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.737415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.737783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.739301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:06.740773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:06.743079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 131/272] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:06.748447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:06.832859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459066-f4ed-4c29-9117-c08f02cbd3fe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:06.873385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:06.874227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.874513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.874785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.878779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1be6aa-fc79-482a-ab7f-9a3924283474 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:06.886248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:06.894899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:06.895817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 142/273] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:06.897008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:06.899649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:06.905562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:06.929084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:06.930494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:06.931280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:06.931771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:06.933556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:06.935865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:06.937984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 132/274] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:06 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:06.939907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:07.013405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3b61d-1be8-4ac8-a819-f8f6eb27db20 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.056874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.057608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.057949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.058235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.058975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:07.061260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:07.068609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40758ed1-581d-4871-aa90-9f47edc25df2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.089308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 143/275] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:06 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:07.090498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:07.109769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5286d-8b1b-4c6a-8024-905be087a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.110818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5286d-8b1b-4c6a-8024-905be087a291 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.111336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5286d-8b1b-4c6a-8024-905be087a291 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.111819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5286d-8b1b-4c6a-8024-905be087a291 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.112760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5286d-8b1b-4c6a-8024-905be087a291 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:07.113360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5286d-8b1b-4c6a-8024-905be087a291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:07.115057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5286d-8b1b-4c6a-8024-905be087a291 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:07.125071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 133/276] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:07.127391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914fe2a-4375-4e38-b9eb-adab25dc64d9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.128193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914fe2a-4375-4e38-b9eb-adab25dc64d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.128973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914fe2a-4375-4e38-b9eb-adab25dc64d9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.132050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 134/277] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:07.133923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914fe2a-4375-4e38-b9eb-adab25dc64d9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:07.165805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:07.196201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:07.226116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef5ed-99e8-4af9-87c4-47e17abfbaaa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.255765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914fe2a-4375-4e38-b9eb-adab25dc64d9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.268192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d9bad-dd6e-418f-b56a-2eb7156ea64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.268791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d9bad-dd6e-418f-b56a-2eb7156ea64b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.269141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d9bad-dd6e-418f-b56a-2eb7156ea64b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.269431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d9bad-dd6e-418f-b56a-2eb7156ea64b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.270445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d9bad-dd6e-418f-b56a-2eb7156ea64b None admin] RBAC: Adding query filter params (role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=ef205a80569946beaa49a091ba7df4ad) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:07.270833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d9bad-dd6e-418f-b56a-2eb7156ea64b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:07.272358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d9bad-dd6e-418f-b56a-2eb7156ea64b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:07.280349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 144/278] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:07 2022] GET /identity/v3/role_assignments?user.id=ef205a80569946beaa49a091ba7df4ad&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:07.281798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20eef-b7b4-470a-a0e1-bc4ea5357230 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.282137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20eef-b7b4-470a-a0e1-bc4ea5357230 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.282422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20eef-b7b4-470a-a0e1-bc4ea5357230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.299815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.301522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.302079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.302250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-37d20eef-b7b4-470a-a0e1-bc4ea5357230 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:07.302997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] PATH_INFO: `/v3/users/58b57e1ea8b24573b21515a535aec9b1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.303542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-37d20eef-b7b4-470a-a0e1-bc4ea5357230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:07.319847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] RBAC: Authorizing `identity:get_user(user_id=58b57e1ea8b24573b21515a535aec9b1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:07.321709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:07.324134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 135/279] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:07 2022] GET /identity/v3/users/58b57e1ea8b24573b21515a535aec9b1 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:07.330176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:07.387962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 145/280] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:07.415057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20eef-b7b4-470a-a0e1-bc4ea5357230 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:07.449706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea480bb0-78a1-428d-82c2-a80b2052670e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.459210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:07.485276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:07.492627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.493565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.493973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.494344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.495278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, user.id=58b57e1ea8b24573b21515a535aec9b1, scope.system=all) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:07.495688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:07.497235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:07.503640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 136/281] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Jun 17 11:25:07 2022] GET /identity/v3/role_assignments?user.id=58b57e1ea8b24573b21515a535aec9b1&scope.system=all&role.id=9ebaaf52f4cd402880b403c50600168e => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:07.546000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20eef-b7b4-470a-a0e1-bc4ea5357230 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.586762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.587853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.588292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.588721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.603744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:07.612187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:07.613170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:07.614137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 146/282] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:07.621437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:07.753114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d72d-ad92-451c-9f1b-da49d7169ca9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.788702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.789737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.790061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.790339 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.791096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:07.791408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:07.792786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:07.802742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 137/283] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:07.808185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:07.915020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bc156-8be4-48d2-88d0-8cea0bf83eba None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:07.930189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:07.950861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:07.951780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:07.952068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:07.952345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] PATH_INFO: `/v3/users/d0dca047615643dabd79fa05ef786402` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:07.953563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] RBAC: Authorizing `identity:get_user(user_id=d0dca047615643dabd79fa05ef786402)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:07.955054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:07.957004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 147/284] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:07 2022] GET /identity/v3/users/d0dca047615643dabd79fa05ef786402 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:07.962511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:08.000281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:08.083615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160e4f9-c500-4fe3-af99-0cc9aaf3fd88 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:08.119009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3ad1e-8789-493e-b50b-888cb0e1681d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:08.119884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3ad1e-8789-493e-b50b-888cb0e1681d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:08.120168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3ad1e-8789-493e-b50b-888cb0e1681d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:08.120446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3ad1e-8789-493e-b50b-888cb0e1681d None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:08.121642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3ad1e-8789-493e-b50b-888cb0e1681d None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:08.123279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3ad1e-8789-493e-b50b-888cb0e1681d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:08.125312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 138/285] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:07 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:08.131017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:08.243140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6aceb-403e-48fd-952d-ab7a4710a442 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:08.284907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cb967-6596-49e4-b9c5-48ff7d7fbe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:08.286203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb967-6596-49e4-b9c5-48ff7d7fbe2f None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:08.286757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb967-6596-49e4-b9c5-48ff7d7fbe2f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:08.287230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb967-6596-49e4-b9c5-48ff7d7fbe2f None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/d0dca047615643dabd79fa05ef786402/roles/21c858abff6f4b9c90786fbcec903396` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:08.295705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cb967-6596-49e4-b9c5-48ff7d7fbe2f None admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3bce33fec47b5b23c70bedccc7dc9, user_id=d0dca047615643dabd79fa05ef786402, role_id=21c858abff6f4b9c90786fbcec903396)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:08.297364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cb967-6596-49e4-b9c5-48ff7d7fbe2f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:08.320042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 148/286] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:08 2022] PUT /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/users/d0dca047615643dabd79fa05ef786402/roles/21c858abff6f4b9c90786fbcec903396 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:08.397225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f1d1-b100-4140-ad55-a521ed066d11 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:08.398321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f1d1-b100-4140-ad55-a521ed066d11 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:08.398842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f1d1-b100-4140-ad55-a521ed066d11 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:08.400535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 139/287] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:08.406709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176dd62-dcd9-4313-a70d-49a68566fe70 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:08.407339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176dd62-dcd9-4313-a70d-49a68566fe70 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:08.407818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176dd62-dcd9-4313-a70d-49a68566fe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:08.430973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c176dd62-dcd9-4313-a70d-49a68566fe70 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:08.432208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c176dd62-dcd9-4313-a70d-49a68566fe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:08.484304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:08.525081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 149/288] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:08.552014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f1d1-b100-4140-ad55-a521ed066d11 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:08.693817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:08.698386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f1d1-b100-4140-ad55-a521ed066d11 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:08.747576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:08.748434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:08.748721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:08.748997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:08.769519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:08.777295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:08.778215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:08.779571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 140/289] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:08.786156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176dd62-dcd9-4313-a70d-49a68566fe70 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:08.911648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176dd62-dcd9-4313-a70d-49a68566fe70 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:08.946417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:08.947027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:08.947312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:08.947579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:08.948342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:08.948655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:08.950059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:08.959309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 150/290] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:08.967515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:09.124545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b5e79-e697-49dd-b340-add7adac84fb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:09.184842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:09.185761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:09.186075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:09.186353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] PATH_INFO: `/v3/users/85f0126e7b1e4980835af0da6f34e1ee` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:09.190552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] RBAC: Authorizing `identity:get_user(user_id=85f0126e7b1e4980835af0da6f34e1ee)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:09.192017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:09.193985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 141/291] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:08 2022] GET /identity/v3/users/85f0126e7b1e4980835af0da6f34e1ee => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:09.199512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:09.258910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:09.347219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142f5fa-09a1-4d36-b472-d0424ca51ec2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:09.401336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:09.402029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:09.402316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:09.402596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:09.501599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:09.503109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:09.547165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:09.654244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 151/292] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:09.660062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:09.708509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:09.816073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa48e3a-eb5a-43b3-a736-53a90a663011 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:09.861373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adac158-d55a-4a43-8890-4e83884d72df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:09.862289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adac158-d55a-4a43-8890-4e83884d72df None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:09.862578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adac158-d55a-4a43-8890-4e83884d72df None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:09.862859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adac158-d55a-4a43-8890-4e83884d72df None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:09.864069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adac158-d55a-4a43-8890-4e83884d72df None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:09.865524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adac158-d55a-4a43-8890-4e83884d72df None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:09.867575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 142/293] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:09 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:09.874107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:10.032422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e9d78-5131-4a58-b5c7-a0a54cbd819e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:10.064696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:10.077433 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6ec76-cf99-46f4-a8bf-01259edc6abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:10.078441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6ec76-cf99-46f4-a8bf-01259edc6abd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.078963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6ec76-cf99-46f4-a8bf-01259edc6abd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.079432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6ec76-cf99-46f4-a8bf-01259edc6abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.080472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6ec76-cf99-46f4-a8bf-01259edc6abd None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=85f0126e7b1e4980835af0da6f34e1ee) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:10.080984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6ec76-cf99-46f4-a8bf-01259edc6abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:10.082632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6ec76-cf99-46f4-a8bf-01259edc6abd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:10.089292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 152/294] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:09 2022] GET /identity/v3/role_assignments?user.id=85f0126e7b1e4980835af0da6f34e1ee&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=9ebaaf52f4cd402880b403c50600168e => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:10.197172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98328950-68e2-4cc1-a3ad-e18ef3c6d320 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.197969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98328950-68e2-4cc1-a3ad-e18ef3c6d320 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.198448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98328950-68e2-4cc1-a3ad-e18ef3c6d320 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.200269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 143/295] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:10.206009 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b908a3-fe40-47f6-ba85-edf26c01f77b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.206551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b908a3-fe40-47f6-ba85-edf26c01f77b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.207021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b908a3-fe40-47f6-ba85-edf26c01f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.231106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a4b908a3-fe40-47f6-ba85-edf26c01f77b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:10.233346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a4b908a3-fe40-47f6-ba85-edf26c01f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:10.336789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 153/296] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:10.350637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b386b-0368-40e4-8fe4-95351264f98a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.350921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b386b-0368-40e4-8fe4-95351264f98a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.351196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b386b-0368-40e4-8fe4-95351264f98a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.352578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 144/297] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:10.364153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b908a3-fe40-47f6-ba85-edf26c01f77b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:10.364683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c102-39fc-4395-8bfa-8c6d55560602 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.364835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c102-39fc-4395-8bfa-8c6d55560602 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.365340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9c102-39fc-4395-8bfa-8c6d55560602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.383680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6ca9c102-39fc-4395-8bfa-8c6d55560602 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:10.385238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6ca9c102-39fc-4395-8bfa-8c6d55560602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:10.467084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 145/298] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:10.468820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44552905-87ef-4ca5-87fa-6f405fe1199d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.469316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44552905-87ef-4ca5-87fa-6f405fe1199d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.469795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44552905-87ef-4ca5-87fa-6f405fe1199d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.471333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 146/299] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:10.477968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd68f3c-29f8-4b39-9028-7f9f1242ada0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.478257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd68f3c-29f8-4b39-9028-7f9f1242ada0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.478702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd68f3c-29f8-4b39-9028-7f9f1242ada0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.496564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3cd68f3c-29f8-4b39-9028-7f9f1242ada0 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:10.497805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3cd68f3c-29f8-4b39-9028-7f9f1242ada0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:10.499124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b908a3-fe40-47f6-ba85-edf26c01f77b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:10.527776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:10.531451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:10.542284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:10.543406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.543886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.544376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.562075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:10.571860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:10.572752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:10.573528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 154/300] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:10.574280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:10.593251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 147/301] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:10.594810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd68f3c-29f8-4b39-9028-7f9f1242ada0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:10.682221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874815-e1ec-4ebb-a463-6d104c297190 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:10.715262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd68f3c-29f8-4b39-9028-7f9f1242ada0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:10.721057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:10.721880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.722172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.722447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.734958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:10.742683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:10.743532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:10.744310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 155/302] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:10.744949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:10.757880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:10.758566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.758855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.759136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.759898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:10.760238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:10.761595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:10.775025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 148/303] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:10.775025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:10.797886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:10.798288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:10.868690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09d16-5339-483a-91a4-6c0ceba2ad40 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:10.884138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cf058-22e4-4559-84fb-f54ac6271872 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:10.905807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:10.906714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.907005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.907310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.908833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:10.910364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:10.916707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 156/304] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:10.916707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:10.924645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:10.925437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:10.925712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:10.926015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:10.926774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:10.927080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:10.928907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:10.942202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 149/305] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:10.942202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:11.027865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:11.032531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55dff4e-cc60-4e96-91af-a322b2600025 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.048966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81644969-2e57-4b38-8fd5-fcb1d76cb89d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.050853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:11.075195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.076682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.076974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.077256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] PATH_INFO: `/v3/users/f5407d0996424be2a4c0196dd6dcc7c8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.090859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.091614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.092227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.092227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] PATH_INFO: `/v3/groups` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.092875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:11.094411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:11.095212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] RBAC: Authorizing `identity:get_user(user_id=f5407d0996424be2a4c0196dd6dcc7c8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:11.096688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:11.098489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 157/306] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:10 2022] GET /identity/v3/users/f5407d0996424be2a4c0196dd6dcc7c8 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:11.098489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:11.114805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 150/307] 10.222.0.25 () {60 vars in 1134 bytes} [Fri Jun 17 11:25:10 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:11.127018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:11.222304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8b030-b37f-43f8-a3eb-36ba79375d8a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.250081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c781c-44cf-4a3e-80b8-289ae4a431b4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.270170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010916-fb2b-40b4-9dc8-d903e4117e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.271060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010916-fb2b-40b4-9dc8-d903e4117e6f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.271340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010916-fb2b-40b4-9dc8-d903e4117e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.271611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010916-fb2b-40b4-9dc8-d903e4117e6f None admin] PATH_INFO: `/v3/users/58b57e1ea8b24573b21515a535aec9b1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.272829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010916-fb2b-40b4-9dc8-d903e4117e6f None admin] RBAC: Authorizing `identity:get_user(user_id=58b57e1ea8b24573b21515a535aec9b1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:11.274998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010916-fb2b-40b4-9dc8-d903e4117e6f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:11.281960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 158/308] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:11 2022] GET /identity/v3/users/58b57e1ea8b24573b21515a535aec9b1 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:11.281960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de942d40-5fb7-4d89-9b1c-382405ada289 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.281960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de942d40-5fb7-4d89-9b1c-382405ada289 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.281960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de942d40-5fb7-4d89-9b1c-382405ada289 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.281960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 159/309] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:11.282340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de942d40-5fb7-4d89-9b1c-382405ada289 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:11.290893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e7db6-3d2d-46ed-aec1-b0ddff5a2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.291571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7db6-3d2d-46ed-aec1-b0ddff5a2324 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.292582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7db6-3d2d-46ed-aec1-b0ddff5a2324 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.292582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e7db6-3d2d-46ed-aec1-b0ddff5a2324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.292946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e7db6-3d2d-46ed-aec1-b0ddff5a2324 None admin] RBAC: Adding query filter params (role.id=21c858abff6f4b9c90786fbcec903396, user.id=f5407d0996424be2a4c0196dd6dcc7c8, scope.system=all) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:11.293321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e7db6-3d2d-46ed-aec1-b0ddff5a2324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:11.295314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e7db6-3d2d-46ed-aec1-b0ddff5a2324 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:11.301328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 151/310] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Jun 17 11:25:11 2022] GET /identity/v3/role_assignments?user.id=f5407d0996424be2a4c0196dd6dcc7c8&scope.system=all&role.id=21c858abff6f4b9c90786fbcec903396 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:11.302858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8cbe-9b2b-4bf6-b353-389988fc015f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8cbe-9b2b-4bf6-b353-389988fc015f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.303422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8cbe-9b2b-4bf6-b353-389988fc015f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.322251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:11.326622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6ddd8cbe-9b2b-4bf6-b353-389988fc015f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:11.328046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6ddd8cbe-9b2b-4bf6-b353-389988fc015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:11.339724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:11.421110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 152/311] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:11.426767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de942d40-5fb7-4d89-9b1c-382405ada289 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.445741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd8cbe-9b2b-4bf6-b353-389988fc015f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:11.469495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.470675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.471102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.471522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] PATH_INFO: `/v3/system/users/58b57e1ea8b24573b21515a535aec9b1/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.473249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=58b57e1ea8b24573b21515a535aec9b1, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:11.474829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:11.485488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 160/312] 10.222.0.25 () {58 vars in 1334 bytes} [Fri Jun 17 11:25:11 2022] PUT /identity/v3/system/users/58b57e1ea8b24573b21515a535aec9b1/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:25:11.561196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd8cbe-9b2b-4bf6-b353-389988fc015f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.600131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.601033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.601317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.601590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.603753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:11.616015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:11.624084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:11.625045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:11.625827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 153/313] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:11.632316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:11.751078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257406-e3a6-4b0a-9a16-16fd93f9cf1b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.753580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:11.789516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.790505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.790926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.791339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.792223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:11.792683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:11.794209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:11.808374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 161/314] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:11.809929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:11.915455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f4a88-9b55-40ba-8481-8857e7c2019a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:11.955471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:11.956357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:11.956641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:11.956917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] PATH_INFO: `/v3/users/d0dca047615643dabd79fa05ef786402` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:11.958152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] RBAC: Authorizing `identity:get_user(user_id=d0dca047615643dabd79fa05ef786402)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:11.959613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:11.961249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 154/315] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:11 2022] GET /identity/v3/users/d0dca047615643dabd79fa05ef786402 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:11.968822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:12.076917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef262-fca0-4590-86bb-0483f7d03e02 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:12.111856 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:12.112859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:12.113344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:12.113809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:12.150587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:12.185822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:12.187654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:12.290544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 162/316] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:12.301952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:12.374080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:12.415633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dafb8-6338-445c-ab7b-ef65adf278a1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:12.450607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8245a64-88e0-4cd4-9245-4ddb8b22c065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:12.452039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8245a64-88e0-4cd4-9245-4ddb8b22c065 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:12.452321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8245a64-88e0-4cd4-9245-4ddb8b22c065 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:12.452617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8245a64-88e0-4cd4-9245-4ddb8b22c065 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:12.453795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8245a64-88e0-4cd4-9245-4ddb8b22c065 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:12.455275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8245a64-88e0-4cd4-9245-4ddb8b22c065 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:12.457289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 155/317] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:12 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:12.463239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:12.538387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:12.579417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a093-d0e2-4804-a8a3-d152b30a9370 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:12.614513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e763c3-b61d-4509-8eb7-3ddf8d63129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:12.615513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e763c3-b61d-4509-8eb7-3ddf8d63129d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:12.616003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e763c3-b61d-4509-8eb7-3ddf8d63129d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:12.616465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e763c3-b61d-4509-8eb7-3ddf8d63129d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:12.617484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e763c3-b61d-4509-8eb7-3ddf8d63129d None admin] RBAC: Adding query filter params (role.id=21c858abff6f4b9c90786fbcec903396, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9, user.id=d0dca047615643dabd79fa05ef786402) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:12.618026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e763c3-b61d-4509-8eb7-3ddf8d63129d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:12.619649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e763c3-b61d-4509-8eb7-3ddf8d63129d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:12.627124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 163/318] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:12 2022] GET /identity/v3/role_assignments?user.id=d0dca047615643dabd79fa05ef786402&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=21c858abff6f4b9c90786fbcec903396 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:13.384123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382feaf-792b-49d0-9368-7e74b82ab59f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.384635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382feaf-792b-49d0-9368-7e74b82ab59f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.384711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382feaf-792b-49d0-9368-7e74b82ab59f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.386100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 156/319] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:13.391558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d7912-da1d-4833-92a4-8feb94d7a0e2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.391927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d7912-da1d-4833-92a4-8feb94d7a0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.392276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d7912-da1d-4833-92a4-8feb94d7a0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.409920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-da3d7912-da1d-4833-92a4-8feb94d7a0e2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:13.411127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-da3d7912-da1d-4833-92a4-8feb94d7a0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:13.480863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 164/320] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:13.503393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382feaf-792b-49d0-9368-7e74b82ab59f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:13.596407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ce116-a865-414a-b4b0-b8b54cf11120 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.596925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ce116-a865-414a-b4b0-b8b54cf11120 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.597200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ce116-a865-414a-b4b0-b8b54cf11120 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.598878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 165/321] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:13.603990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00b60-8d4c-4bce-9f23-f934fe28db12 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.604281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00b60-8d4c-4bce-9f23-f934fe28db12 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.604556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a00b60-8d4c-4bce-9f23-f934fe28db12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.611918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382feaf-792b-49d0-9368-7e74b82ab59f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:13.621250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-79a00b60-8d4c-4bce-9f23-f934fe28db12 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:13.622702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-79a00b60-8d4c-4bce-9f23-f934fe28db12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:13.652310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:13.661542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:13.662414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.662962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.663238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.664463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:13.665950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:13.667773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 157/322] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:13.673214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:13.688166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:13.701463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 166/323] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:13.712255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8cb39-b7ae-47ef-b244-67cdd0faea48 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.712553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8cb39-b7ae-47ef-b244-67cdd0faea48 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.712835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8cb39-b7ae-47ef-b244-67cdd0faea48 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.714330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 167/324] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:13.719542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86f95-b082-46bd-903d-1b6eb801f91f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.719831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86f95-b082-46bd-903d-1b6eb801f91f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.720390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86f95-b082-46bd-903d-1b6eb801f91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.736326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-91c86f95-b082-46bd-903d-1b6eb801f91f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:13.737450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-91c86f95-b082-46bd-903d-1b6eb801f91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:13.776853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c4a9c-6403-48da-9274-cf85f796ec17 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:13.804262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 168/325] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:13.805282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c86f95-b082-46bd-903d-1b6eb801f91f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:13.812549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:13.813223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.813511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.813793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] PATH_INFO: `/v3/groups` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.814531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:13.816393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:13.838144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 158/326] 10.222.0.25 () {60 vars in 1134 bytes} [Fri Jun 17 11:25:13 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:25:13.838144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:13.905154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:13.922277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c86f95-b082-46bd-903d-1b6eb801f91f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:13.945025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:13.960162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44062a8-20e5-4383-9cc3-64e9d1d0bfa1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:13.960622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:13.961453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:13.961747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:13.962053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:13.973393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:13.981314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:13.982557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:13.983329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 169/327] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:13.988630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:14.005375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:14.006898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:14.007004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:14.007364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:14.019876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:14.028472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:14.029593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:14.030662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:14.031419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 159/328] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:14.037978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:14.112371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310f185-a9dc-429b-b9b9-8282d05a7723 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:14.156166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:14.156771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:14.157048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:14.157317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:14.158268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:14.158582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:14.159965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:14.165340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813ea3-b714-4bb0-9a86-45dc43a8aead None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:14.170074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 170/329] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:14.179019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:14.202415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:14.207506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:14.208212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:14.208496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:14.209466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:14.209581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:14.209819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:14.211240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:14.224772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 160/330] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:14.226054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:14.230837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:14.313311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d4ad4-4403-4a72-a7c0-0866438d2fd3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:14.344304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4116980-afa9-4252-8a2c-e271ced212e2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:14.353165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:14.354460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:14.354964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:14.355534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] PATH_INFO: `/v3/users/f5407d0996424be2a4c0196dd6dcc7c8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:14.358094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] RBAC: Authorizing `identity:get_user(user_id=f5407d0996424be2a4c0196dd6dcc7c8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:14.359738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:14.362183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 171/331] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:14 2022] GET /identity/v3/users/f5407d0996424be2a4c0196dd6dcc7c8 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:14.370948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:14.384661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:14.386154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:14.386440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:14.386712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] PATH_INFO: `/v3/users/58b57e1ea8b24573b21515a535aec9b1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:14.388158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] RBAC: Authorizing `identity:get_user(user_id=58b57e1ea8b24573b21515a535aec9b1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:14.389808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:14.393058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 161/332] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:14 2022] GET /identity/v3/users/58b57e1ea8b24573b21515a535aec9b1 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:14.398974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:14.463289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:14.526130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88463-127a-4dc9-b4a7-da9888908c4e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:14.526665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c067e6b-65a2-4f8d-8cf8-26ba949c26fd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:14.538606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:14.566423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb4799-069f-4741-b157-28a67c27c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:14.567136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb4799-069f-4741-b157-28a67c27c414 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:14.567423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb4799-069f-4741-b157-28a67c27c414 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:14.567697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb4799-069f-4741-b157-28a67c27c414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:14.568534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb4799-069f-4741-b157-28a67c27c414 None admin] RBAC: Adding query filter params (role.id=9ebaaf52f4cd402880b403c50600168e, user.id=58b57e1ea8b24573b21515a535aec9b1, scope.system=all) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:14.568843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb4799-069f-4741-b157-28a67c27c414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:14.570699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb4799-069f-4741-b157-28a67c27c414 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:14.572276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd261b2e-9bea-4c97-862b-9442055cbd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:14.573162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd261b2e-9bea-4c97-862b-9442055cbd48 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:14.573450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd261b2e-9bea-4c97-862b-9442055cbd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:14.573731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd261b2e-9bea-4c97-862b-9442055cbd48 None admin] PATH_INFO: `/v3/system/users/f5407d0996424be2a4c0196dd6dcc7c8/roles/21c858abff6f4b9c90786fbcec903396` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:14.576429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd261b2e-9bea-4c97-862b-9442055cbd48 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f5407d0996424be2a4c0196dd6dcc7c8, role_id=21c858abff6f4b9c90786fbcec903396)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:14.578092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd261b2e-9bea-4c97-862b-9442055cbd48 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:14.586489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 162/333] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Jun 17 11:25:14 2022] GET /identity/v3/role_assignments?user.id=58b57e1ea8b24573b21515a535aec9b1&scope.system=all&role.id=9ebaaf52f4cd402880b403c50600168e => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:14.588796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 172/334] 10.222.0.25 () {58 vars in 1334 bytes} [Fri Jun 17 11:25:14 2022] PUT /identity/v3/system/users/f5407d0996424be2a4c0196dd6dcc7c8/roles/21c858abff6f4b9c90786fbcec903396 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:16.521297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c0cf7-fe43-49dd-8cc7-8298f0690fe9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:16.521927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c0cf7-fe43-49dd-8cc7-8298f0690fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:16.521927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c0cf7-fe43-49dd-8cc7-8298f0690fe9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:16.523533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 163/335] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:16.529410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4dd8d-2c46-41c6-a8b5-62c318c75c53 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:16.529925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4dd8d-2c46-41c6-a8b5-62c318c75c53 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:16.530099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4dd8d-2c46-41c6-a8b5-62c318c75c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:16.549981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-49c4dd8d-2c46-41c6-a8b5-62c318c75c53 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:16.551159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-49c4dd8d-2c46-41c6-a8b5-62c318c75c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:16.633139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 173/336] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:16.667929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c0cf7-fe43-49dd-8cc7-8298f0690fe9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:16.794759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c0cf7-fe43-49dd-8cc7-8298f0690fe9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:16.802156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:16.841318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:16.842191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:16.842478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:16.842754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:16.854094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ff670-c794-4978-b214-367f3231b681 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:16.854423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ff670-c794-4978-b214-367f3231b681 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:16.854653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ff670-c794-4978-b214-367f3231b681 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:16.856390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 174/337] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:16.858249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:16.862260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255436ba-092d-46ba-bbb1-0ca0e6d52bef None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:16.862554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255436ba-092d-46ba-bbb1-0ca0e6d52bef None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:16.862839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255436ba-092d-46ba-bbb1-0ca0e6d52bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:16.866287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:16.868336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:16.870351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 164/338] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:16.878046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:16.902455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-255436ba-092d-46ba-bbb1-0ca0e6d52bef None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:16.903711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-255436ba-092d-46ba-bbb1-0ca0e6d52bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:17.016697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 175/339] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:17.018075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44978597-4d4b-4312-8652-29fb1fd5e83a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.018373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44978597-4d4b-4312-8652-29fb1fd5e83a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.018649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44978597-4d4b-4312-8652-29fb1fd5e83a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.020199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 176/340] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:17.025731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a58ed-4eee-4859-98bc-e375d01cb30f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.026167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a58ed-4eee-4859-98bc-e375d01cb30f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.026517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a58ed-4eee-4859-98bc-e375d01cb30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.054885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-998a58ed-4eee-4859-98bc-e375d01cb30f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:17.056012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-998a58ed-4eee-4859-98bc-e375d01cb30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:17.064463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0e79f-3d19-4998-879f-035dcbcb082c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:17.118086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:17.150442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:17.151499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.151980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.152449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.153436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:17.154011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:17.155618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:17.173287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 165/341] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:17.174490 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:17.182092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 177/342] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:17.184904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a58ed-4eee-4859-98bc-e375d01cb30f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:17.252195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:17.295001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a58ed-4eee-4859-98bc-e375d01cb30f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:17.331655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:17.332245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.332531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.332807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.366402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:17.405888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e917b-6e65-457e-a0dd-c2859783307c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:17.407080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:17.408570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:17.468126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:17.468956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.469236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.469508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.494375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:17.503653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:17.504513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:17.505292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:17.510318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 166/343] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:17.511813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:17.514932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 178/344] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:25:17.520240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:17.646482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:17.689066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e756bf6-df75-434e-9031-1b5e2cec944e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:17.732339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:17.733558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.734020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.734459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.736077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:17.737734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:17.740809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 179/345] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:17 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:17.741145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:17.742335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:17.780265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba577d0-f893-493d-b9d9-217da7700726 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:17.845794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:17.846804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.847094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.847370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.854022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eaf4f0-b4ce-4bcf-ade7-09930e0692ec None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:17.872773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:17.880906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:17.882052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:17.883152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:17.887124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 167/346] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:17.888448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:17.890188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:17.895985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea524339-3646-4672-9b94-ceb47130edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:17.896921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea524339-3646-4672-9b94-ceb47130edb8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.897402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea524339-3646-4672-9b94-ceb47130edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.897855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea524339-3646-4672-9b94-ceb47130edb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.898759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea524339-3646-4672-9b94-ceb47130edb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:17.899216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea524339-3646-4672-9b94-ceb47130edb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:17.900807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea524339-3646-4672-9b94-ceb47130edb8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:17.910712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 180/347] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:17.912232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9ff7b-a51b-4502-b0e2-f1e07e2ae0b5 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:17.912932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9ff7b-a51b-4502-b0e2-f1e07e2ae0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:17.913353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9ff7b-a51b-4502-b0e2-f1e07e2ae0b5 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:17.914792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 181/348] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:17.915707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9ff7b-a51b-4502-b0e2-f1e07e2ae0b5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:18.029002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9ff7b-a51b-4502-b0e2-f1e07e2ae0b5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:18.039790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6bff7-e722-4232-838a-c0bd0343c350 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:18.072390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:18.073078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.073364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.073646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.074815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:18.075189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:18.076953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:18.087192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fd929-6cd4-4383-b6b5-c84de753edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:18.088309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd929-6cd4-4383-b6b5-c84de753edb8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.088762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd929-6cd4-4383-b6b5-c84de753edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.088941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 182/349] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:18.089329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd929-6cd4-4383-b6b5-c84de753edb8 None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.089861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:18.099015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd929-6cd4-4383-b6b5-c84de753edb8 None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:18.100708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd929-6cd4-4383-b6b5-c84de753edb8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:18.102639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 168/350] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:17 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:18.104056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35012d7d-481e-4035-ada2-bec2eac5fbea None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.104552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35012d7d-481e-4035-ada2-bec2eac5fbea None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.105053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35012d7d-481e-4035-ada2-bec2eac5fbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.124885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-35012d7d-481e-4035-ada2-bec2eac5fbea None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:18.126428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-35012d7d-481e-4035-ada2-bec2eac5fbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:18.138273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:18.163515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:18.215410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 169/351] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:18.217168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35012d7d-481e-4035-ada2-bec2eac5fbea None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:18.219338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf18d7-6ce1-4b68-b0cf-805b8e7a3abe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:18.267745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:18.269233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.269725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.270263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.345301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35012d7d-481e-4035-ada2-bec2eac5fbea None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:18.366211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:18.368621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:18.387224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:18.389452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.389693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.390983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.407715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:18.442471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:18.487478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:18.488956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:18.502733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 183/352] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:18.503296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:18.622757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd626-191d-48e6-879f-c23f83d360e0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:18.650909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 170/353] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:18.651631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:18.668151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bff7a9-802c-4d58-9e9e-8d3a7bf68dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:18.669510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff7a9-802c-4d58-9e9e-8d3a7bf68dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.670033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff7a9-802c-4d58-9e9e-8d3a7bf68dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.670477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff7a9-802c-4d58-9e9e-8d3a7bf68dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.671465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bff7a9-802c-4d58-9e9e-8d3a7bf68dd6 None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=577d26cace0e4d3fba244021e34a6d6b) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:18.671919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bff7a9-802c-4d58-9e9e-8d3a7bf68dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:18.673480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bff7a9-802c-4d58-9e9e-8d3a7bf68dd6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:18.680894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 184/354] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:18 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=9ebaaf52f4cd402880b403c50600168e => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:18.682413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd40fc-c6bf-41c2-a992-4809ebfab779 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.682841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd40fc-c6bf-41c2-a992-4809ebfab779 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.683257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd40fc-c6bf-41c2-a992-4809ebfab779 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.684694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 185/355] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:18.685778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd40fc-c6bf-41c2-a992-4809ebfab779 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:18.713058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:18.717888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:18.795767 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4fc0a-bfb8-4bdc-a158-ae46475127ca None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:18.839796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd40fc-c6bf-41c2-a992-4809ebfab779 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:18.847580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:18.848739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.849863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.850230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.864067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:18.873201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:18.874265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 171/356] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:18.875616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:18.886062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86073f-65e4-4ca0-866c-3178c3676331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:18.888748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86073f-65e4-4ca0-866c-3178c3676331 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.889037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86073f-65e4-4ca0-866c-3178c3676331 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.889317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86073f-65e4-4ca0-866c-3178c3676331 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.890732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86073f-65e4-4ca0-866c-3178c3676331 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:18.892200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86073f-65e4-4ca0-866c-3178c3676331 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:18.894482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 186/357] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:18 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:18.895477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db0f1-8856-430b-af94-d362b2358e58 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:18.895767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db0f1-8856-430b-af94-d362b2358e58 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:18.896042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db0f1-8856-430b-af94-d362b2358e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:18.917684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-218db0f1-8856-430b-af94-d362b2358e58 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:18.919497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-218db0f1-8856-430b-af94-d362b2358e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:19.001510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc77f18-be71-4f8d-83d4-dfc26d040b3a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.001955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 187/358] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:19.003253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8c451-67a4-4f9a-9707-436110f110dd None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.003552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8c451-67a4-4f9a-9707-436110f110dd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.003830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8c451-67a4-4f9a-9707-436110f110dd None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.005115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 188/359] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:19.006852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:19.007213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8c451-67a4-4f9a-9707-436110f110dd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.029683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:19.059182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.059652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.060483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.060762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.065347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:19.065347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:19.065638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 172/360] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:18 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:19.067078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.145130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8c451-67a4-4f9a-9707-436110f110dd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.190603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54047d1-6a0c-421c-ad76-a93609a52731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.191584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54047d1-6a0c-421c-ad76-a93609a52731 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.192557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54047d1-6a0c-421c-ad76-a93609a52731 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.193032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54047d1-6a0c-421c-ad76-a93609a52731 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.194005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54047d1-6a0c-421c-ad76-a93609a52731 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:19.194504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54047d1-6a0c-421c-ad76-a93609a52731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:19.196080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54047d1-6a0c-421c-ad76-a93609a52731 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:19.205543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 189/361] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:19.206641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32664-c939-4ff9-aa22-0957fda20956 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.206841 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b207d-7a74-45a9-9360-cf3b5132b3a7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.206939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32664-c939-4ff9-aa22-0957fda20956 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.207196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32664-c939-4ff9-aa22-0957fda20956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.230295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9aa32664-c939-4ff9-aa22-0957fda20956 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:19.231592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9aa32664-c939-4ff9-aa22-0957fda20956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:19.250006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.251095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.251436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.251716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.253261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:19.254738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:19.257158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 173/362] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:19.262793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.311610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:19.322442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 190/363] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:19.323438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32664-c939-4ff9-aa22-0957fda20956 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.340771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:19.389663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec223f-148d-45c8-878f-10763d2f48a5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.433324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.433324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.433324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.433678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.449634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:19.457727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:19.458735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:19.459579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 174/364] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:19.460270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.477558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32664-c939-4ff9-aa22-0957fda20956 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.525504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.526681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.527460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.527747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.529078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:19.530644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:19.532614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 191/365] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:19.533411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.590474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:19.595303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf13f5-f7d7-4785-9e0c-8a4f0def566c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.643438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.644328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.644612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.644888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] PATH_INFO: `/v3/users/f5407d0996424be2a4c0196dd6dcc7c8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.648566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] RBAC: Authorizing `identity:get_user(user_id=f5407d0996424be2a4c0196dd6dcc7c8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:19.648566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:19.649269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 175/366] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/users/f5407d0996424be2a4c0196dd6dcc7c8 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:19.649999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.660634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:19.701811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9284a-3b24-4444-9ebd-9ce2e81bd522 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.744379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.745651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.746025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.747147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.748583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=577d26cace0e4d3fba244021e34a6d6b) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:19.748905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:19.750398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:19.757533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 192/367] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:19.758360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.799426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92f69-fba3-4c75-9bb2-8a68648028a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.839865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.840796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.841628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.842489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.854835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:19.863996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:19.864939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:19.865724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 176/368] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:19.866443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.911288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:19.913250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e394e-96e6-4f7b-b3ba-796cf13b1102 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:19.960501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:19.961112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:19.961391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:19.961880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:19.962480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:19.962794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:19.964157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:19.974887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 193/369] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:19.975740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:19.985507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:20.015837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd012d01-d19b-4d22-b8fc-d4a7dbbb1f01 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:20.068705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:20.069381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:20.069667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:20.069977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:20.070828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:20.071147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:20.072609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:20.079324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 177/370] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=9ebaaf52f4cd402880b403c50600168e => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:20.080245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:20.140085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f7aab-864e-4b74-a081-1d182ded4d9c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:20.184533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:20.185239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:20.185528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:20.185811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:20.186762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] RBAC: Adding query filter params (role.id=21c858abff6f4b9c90786fbcec903396, user.id=f5407d0996424be2a4c0196dd6dcc7c8, scope.system=all) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:20.187083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:20.188666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:20.203956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 194/371] 10.222.0.25 () {56 vars in 1308 bytes} [Fri Jun 17 11:25:19 2022] GET /identity/v3/role_assignments?user.id=f5407d0996424be2a4c0196dd6dcc7c8&scope.system=all&role.id=21c858abff6f4b9c90786fbcec903396 => generated 510 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:20.204816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:20.211114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:20.226692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d642-2e7c-46fb-989b-4a1bd19a10fb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:20.279853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:20.281426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:20.282212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:20.282732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:20.283930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:20.284887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:20.287039 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:20.287347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:20.299916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 178/372] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:20.310958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:20.400215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af349f-1022-47b4-a624-04f3b941c715 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:20.461266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:20.461874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:20.462156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:20.462425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:20.480672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6976d9c-e8de-43e0-b8fe-523763c13a07 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:20.529599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:20.530310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:20.530750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:20.531042 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:20.563387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:20.563715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:20.564940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:20.628086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:20.631365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:20.650279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:20.724212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 195/373] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:20.729940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:20.837028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 179/374] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:20.843123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:20.867433 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d490e-fe4d-4bbd-8a98-1fb32b4093e7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:20.909205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:20.911983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:20.913492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:20.914547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:20.918085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:20.921059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:20.927970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 196/375] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:20 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:20.933558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:20.956335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:20.991738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:21.037034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991c456-60e8-4101-b762-6e970f2677ed None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:21.096309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff898b-9515-4d7e-8c18-add2e15a4fbe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:21.107129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:21.108174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.108519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.108803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] PATH_INFO: `/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.110170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] RBAC: Authorizing `identity:get_project(project_id=5460a823bf554ac5a99fd5d73c21c6d1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:21.111638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:21.113688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 180/376] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:20 2022] GET /identity/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1 => generated 344 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:21.119105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:21.136115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:21.136990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.137275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.137552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.140473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:21.141996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:21.144041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 197/377] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:20 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:21.149440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:21.262610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6247d91-3992-484d-9172-b84bf532d487 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:21.286838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:21.306600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770f3d1-1dec-4d30-8a72-468cad59e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:21.308051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f3d1-1dec-4d30-8a72-468cad59e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.308726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f3d1-1dec-4d30-8a72-468cad59e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.309373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f3d1-1dec-4d30-8a72-468cad59e7b9 None admin] PATH_INFO: `/v3/groups/f83ebc2986d34eb6b93e7d9c655e6188` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.315984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770f3d1-1dec-4d30-8a72-468cad59e7b9 None admin] RBAC: Authorizing `identity:get_group(group_id=f83ebc2986d34eb6b93e7d9c655e6188)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:21.317857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770f3d1-1dec-4d30-8a72-468cad59e7b9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:21.320693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 181/378] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:21 2022] GET /identity/v3/groups/f83ebc2986d34eb6b93e7d9c655e6188 => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:21.322411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b236a-e06b-4c15-8916-600e64177d8e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.322771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b236a-e06b-4c15-8916-600e64177d8e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.323119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b236a-e06b-4c15-8916-600e64177d8e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.325808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 182/379] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:21.327500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b236a-e06b-4c15-8916-600e64177d8e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:21.359407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:21.365757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbff66-7d36-4d72-be31-e60c78d1390d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:21.409161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3aebb6-34b5-45c3-a2c6-1dfa2e4788ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:21.409825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aebb6-34b5-45c3-a2c6-1dfa2e4788ac None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.410181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aebb6-34b5-45c3-a2c6-1dfa2e4788ac None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.410454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3aebb6-34b5-45c3-a2c6-1dfa2e4788ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.411290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aebb6-34b5-45c3-a2c6-1dfa2e4788ac None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:21.411605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aebb6-34b5-45c3-a2c6-1dfa2e4788ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:21.413286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3aebb6-34b5-45c3-a2c6-1dfa2e4788ac None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:21.419974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 198/380] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:21 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 260 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:21.421188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9f7c-e86b-49f9-81f9-96ef0e1873f4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.421473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9f7c-e86b-49f9-81f9-96ef0e1873f4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.421752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c9f7c-e86b-49f9-81f9-96ef0e1873f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.457212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-555c9f7c-e86b-49f9-81f9-96ef0e1873f4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:21.458449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-555c9f7c-e86b-49f9-81f9-96ef0e1873f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:21.496458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b236a-e06b-4c15-8916-600e64177d8e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:21.587412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 199/381] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:21.590561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:21.591929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.592217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.592497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.593356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] RBAC: Adding query filter params (group.id=f83ebc2986d34eb6b93e7d9c655e6188, role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=5460a823bf554ac5a99fd5d73c21c6d1) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:21.593676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:21.595182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:21.607362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 183/382] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:21 2022] GET /identity/v3/role_assignments?group.id=f83ebc2986d34eb6b93e7d9c655e6188&scope.project.id=5460a823bf554ac5a99fd5d73c21c6d1&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 260 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:21.613551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9f7c-e86b-49f9-81f9-96ef0e1873f4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:21.745371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:21.758706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c9f7c-e86b-49f9-81f9-96ef0e1873f4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:21.800016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:21.800832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.801118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.801456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.814427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:21.823841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:21.824759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:21.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 200/383] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:21.834988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:21.896360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:21.950885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196618b8-a667-4ceb-969b-6f02cd584b3d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:21.986513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:21.987198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:21.987480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:21.987755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:21.988516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:21.988823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:21.990210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:21.999177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 184/384] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:22.004675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:22.129978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a6c05-2a8e-487e-93ac-ad0aeaf2c3c5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:22.172419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:22.173218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:22.173553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:22.173831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:22.175069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:22.176516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:22.178493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 201/385] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:22 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:22.183994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:22.290297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe995c-4b86-475c-b336-7718a3bfbc97 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:22.302219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:22.326561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e807e-0e11-4cd8-9176-8ee8f48ff114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:22.327440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e807e-0e11-4cd8-9176-8ee8f48ff114 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:22.327723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e807e-0e11-4cd8-9176-8ee8f48ff114 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:22.328001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e807e-0e11-4cd8-9176-8ee8f48ff114 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:22.329186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e807e-0e11-4cd8-9176-8ee8f48ff114 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:22.330663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e807e-0e11-4cd8-9176-8ee8f48ff114 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:22.332664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 185/386] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:22 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:22.338416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:22.361705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:22.457286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb3634-9a67-4803-9a34-ddef4a6d4a3a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:22.496155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9c6da-9fe2-4c0a-a6c4-6e9c56849518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:22.497886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9c6da-9fe2-4c0a-a6c4-6e9c56849518 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:22.498176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9c6da-9fe2-4c0a-a6c4-6e9c56849518 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:22.498451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9c6da-9fe2-4c0a-a6c4-6e9c56849518 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:22.500451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9c6da-9fe2-4c0a-a6c4-6e9c56849518 None admin] RBAC: Authorizing `identity:create_grant(project_id=577d26cace0e4d3fba244021e34a6d6b, group_id=18a9de2b95654f0187c3efefff2ef7a7, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:22.501974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9c6da-9fe2-4c0a-a6c4-6e9c56849518 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:22.517648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 202/387] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Jun 17 11:25:22 2022] PUT /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:22.900436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c32d5-d044-4106-a239-1bd0c041e4fc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:22.901377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c32d5-d044-4106-a239-1bd0c041e4fc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:22.901867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c32d5-d044-4106-a239-1bd0c041e4fc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:22.904385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 186/388] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:22.909746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb76e8-f2cc-4bac-9914-73797183f6c7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:22.910705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb76e8-f2cc-4bac-9914-73797183f6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:22.911197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb76e8-f2cc-4bac-9914-73797183f6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:22.931930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5dfb76e8-f2cc-4bac-9914-73797183f6c7 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:22.933478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5dfb76e8-f2cc-4bac-9914-73797183f6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:22.945914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e1566-7950-4eac-a1a6-8e96d9066a96 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:22.946487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e1566-7950-4eac-a1a6-8e96d9066a96 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:22.946957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e1566-7950-4eac-a1a6-8e96d9066a96 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:22.948540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 187/389] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:22.953829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797d83c-36a7-4d6e-93fd-c05f128392d0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:22.954387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797d83c-36a7-4d6e-93fd-c05f128392d0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:22.954881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797d83c-36a7-4d6e-93fd-c05f128392d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:22.977249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f797d83c-36a7-4d6e-93fd-c05f128392d0 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:22.979489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f797d83c-36a7-4d6e-93fd-c05f128392d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:23.068840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 203/390] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:23.100816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb76e8-f2cc-4bac-9914-73797183f6c7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.103523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 188/391] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:23.124599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:23.126294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797d83c-36a7-4d6e-93fd-c05f128392d0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.235582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797d83c-36a7-4d6e-93fd-c05f128392d0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:23.245486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:23.272914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:23.274008 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:23.274429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:23.274859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:23.287891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:23.297011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:23.298171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:23.299139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:23.305994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 189/392] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:23.308186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.407924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb76e8-f2cc-4bac-9914-73797183f6c7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:23.414025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98554fe6-a052-4e44-9281-b04a8f98c334 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:23.450865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:23.451958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:23.452409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:23.452829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:23.453708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:23.454180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:23.456077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:23.469327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 190/393] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:23.471226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.506189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:23.566552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:23.567464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:23.567664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:23.567980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:23.579004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfedee7-1bc3-43a7-8bcb-2d7a8481ebee None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:23.614854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:23.615614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:23.615898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:23.616173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:23.617428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:23.618923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:23.620934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 191/394] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:23 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:23.626907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.630510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:23.646798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:23.647771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:23.648793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 204/395] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 550 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:23.654734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.772128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:23.776355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9fc23-19b2-40d2-91f3-584adac50b2a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:23.814647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:23.815531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:23.815817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:23.816097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:23.817349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:23.818835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:23.820854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 192/396] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:23 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:23.821324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:23.827714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.890291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f05b-73bb-4e29-9854-e8872583d477 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:23.943497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21dbfa-661e-439b-aa52-f60880010f5c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:23.969251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:23.970710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:23.971211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:23.971677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:23.972624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:23.973122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:23.974743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:23.989723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 205/397] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:23.991644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:23.992548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:23.992830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:23.993104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:23.994749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:23.999215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3bce33fec47b5b23c70bedccc7dc9, group_id=18a9de2b95654f0187c3efefff2ef7a7, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:24.000652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:24.032861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 193/398] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Jun 17 11:25:23 2022] PUT /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:24.147876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608aa65-37e2-488a-b851-333b287dba5c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:24.193329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:24.196824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae57b08-ceea-48ec-9673-ec7dea3c74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:24.197721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae57b08-ceea-48ec-9673-ec7dea3c74c3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.198036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae57b08-ceea-48ec-9673-ec7dea3c74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.198311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae57b08-ceea-48ec-9673-ec7dea3c74c3 None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.202740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae57b08-ceea-48ec-9673-ec7dea3c74c3 None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:24.204215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae57b08-ceea-48ec-9673-ec7dea3c74c3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:24.205972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 206/399] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:23 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:24.211512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:24.261988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:24.324540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d33-3739-4843-b99b-2aa9c815909b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.325722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d33-3739-4843-b99b-2aa9c815909b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.326242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b257d33-3739-4843-b99b-2aa9c815909b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.328111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 207/400] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:24.333748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c54e2b-0304-4053-8802-4b17a8e9b8cc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.334336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c54e2b-0304-4053-8802-4b17a8e9b8cc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.334817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c54e2b-0304-4053-8802-4b17a8e9b8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.353184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-06c54e2b-0304-4053-8802-4b17a8e9b8cc None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:24.354771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-06c54e2b-0304-4053-8802-4b17a8e9b8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:24.367060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f95aeb-a9b7-45f1-827a-c00f848e9c9a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:24.429436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 208/401] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:24.440473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:24.441427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.441762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.442073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.446560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:24.448076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:24.450005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 194/402] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:24 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:24.455528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c54e2b-0304-4053-8802-4b17a8e9b8cc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:24.455751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:24.561068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c54e2b-0304-4053-8802-4b17a8e9b8cc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:24.567507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:24.575814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a23d7-4ae1-429e-bcef-5afcd781cd90 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:24.593691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:24.600860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66d6c9-b8b5-4822-b797-5bebfdce02cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:24.602123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d6c9-b8b5-4822-b797-5bebfdce02cf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.602643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d6c9-b8b5-4822-b797-5bebfdce02cf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.603120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66d6c9-b8b5-4822-b797-5bebfdce02cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.614948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66d6c9-b8b5-4822-b797-5bebfdce02cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:24.617247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:24.618163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.618441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.618714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.620983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] RBAC: Authorizing `identity:create_grant(project_id=577d26cace0e4d3fba244021e34a6d6b, group_id=18a9de2b95654f0187c3efefff2ef7a7, role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:24.622483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1d66d6c9-b8b5-4822-b797-5bebfdce02cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:24.624766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:24.625608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:24.626442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:24.629089 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 209/403] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:24.635572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e372c-e269-4d40-962b-8e1d8bd67665 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.636158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e372c-e269-4d40-962b-8e1d8bd67665 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.636658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e372c-e269-4d40-962b-8e1d8bd67665 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.638238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 195/404] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Jun 17 11:25:24 2022] PUT /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:25:24.639863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:24.640069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 210/405] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:24.651499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c2ba2-c823-42b7-bc08-2bcf6938cd6e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.652154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c2ba2-c823-42b7-bc08-2bcf6938cd6e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.652645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c2ba2-c823-42b7-bc08-2bcf6938cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.695025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0e4c2ba2-c823-42b7-bc08-2bcf6938cd6e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:24.696621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0e4c2ba2-c823-42b7-bc08-2bcf6938cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:24.764397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21c2fb-f51f-4dfe-85ff-a74571a0fad6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:24.806526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:24.807568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.808011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.808447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.808660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 211/406] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:24.809667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:24.810145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:24.811796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:24.821203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 196/407] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:24.827531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c2ba2-c823-42b7-bc08-2bcf6938cd6e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:24.839638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:24.867295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:24.941580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c2ba2-c823-42b7-bc08-2bcf6938cd6e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:24.962449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:24.981587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:24.983183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:24.983636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:24.984051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:24.985393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:24.987045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:24.989103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 212/408] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:24 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:24.992257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321ec5f-f415-4ca7-9f51-31499a3f2fd3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:24.994638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.061987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f1acd-8e8e-4911-9d36-99e7940d581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.064251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1acd-8e8e-4911-9d36-99e7940d581d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.065944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1acd-8e8e-4911-9d36-99e7940d581d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.066399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f1acd-8e8e-4911-9d36-99e7940d581d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.084323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f1acd-8e8e-4911-9d36-99e7940d581d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-424f1acd-8e8e-4911-9d36-99e7940d581d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:25.094269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:25.095377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:25.096205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:25.099221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 197/409] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:25.100803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4be13-7c63-4604-807d-6cee7a501b84 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.101224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4be13-7c63-4604-807d-6cee7a501b84 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.101648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4be13-7c63-4604-807d-6cee7a501b84 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.107572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 198/410] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:25.107572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4be13-7c63-4604-807d-6cee7a501b84 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.165802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005f86a-38d7-4907-9aa4-c57db3d4f8ab None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.205493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419eef1-13cc-4fb1-876f-c41507b13a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.206736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419eef1-13cc-4fb1-876f-c41507b13a81 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.207196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419eef1-13cc-4fb1-876f-c41507b13a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.207662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419eef1-13cc-4fb1-876f-c41507b13a81 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.209072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419eef1-13cc-4fb1-876f-c41507b13a81 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:25.210739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419eef1-13cc-4fb1-876f-c41507b13a81 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:25.212941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 213/411] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:24 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:25.214473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef09f-1b7b-41de-869f-4e24d9618993 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.214961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef09f-1b7b-41de-869f-4e24d9618993 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.215475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef09f-1b7b-41de-869f-4e24d9618993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.218209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:25.232589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4be13-7c63-4604-807d-6cee7a501b84 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.233408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-afdef09f-1b7b-41de-869f-4e24d9618993 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:25.234856 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-afdef09f-1b7b-41de-869f-4e24d9618993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:25.280520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.281208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.281495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.281770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.282556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:25.282863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:25.284618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:25.293117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:25.300892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 199/412] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:25.300892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.318741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 214/413] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:25.320003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdef09f-1b7b-41de-869f-4e24d9618993 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.442289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a2f8-8503-421c-89b9-36dd3670222b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.458324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdef09f-1b7b-41de-869f-4e24d9618993 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.485116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.486702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.487323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.487824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.490959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] RBAC: Authorizing `identity:create_grant(project_id=c2e3bce33fec47b5b23c70bedccc7dc9, group_id=18a9de2b95654f0187c3efefff2ef7a7, role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:25.493112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:25.500598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.501768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.502183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.502543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] PATH_INFO: `/v3/groups/f83ebc2986d34eb6b93e7d9c655e6188` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.503990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] RBAC: Authorizing `identity:get_group(group_id=f83ebc2986d34eb6b93e7d9c655e6188)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:25.504998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:25.505877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:25.508186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 215/414] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:25 2022] GET /identity/v3/groups/f83ebc2986d34eb6b93e7d9c655e6188 => generated 229 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:25.509163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.519612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 200/415] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Jun 17 11:25:25 2022] PUT /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9/groups/18a9de2b95654f0187c3efefff2ef7a7/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:25.520978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.582504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:25.645118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72599a65-a058-4cbc-abd5-f16b97e37eee None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.659654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1015a-76d6-453b-9c0e-5c2ce048c7bf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.691048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.692644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.693127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.693602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.705346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.706655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.706977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.707317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:25.707423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] PATH_INFO: `/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.708591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] RBAC: Authorizing `identity:get_project(project_id=5460a823bf554ac5a99fd5d73c21c6d1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:25.710125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:25.712222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 201/416] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:25 2022] GET /identity/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:25.716387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:25.717312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:25.718143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:25.718732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.721078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 216/417] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:25.726883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:25.796835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:25.835248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4330d92-bed2-45dd-ad29-b761811b7617 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.866211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349adda-df7b-47d8-af05-7390ac952c05 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:25.879861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.881100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.881389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.881670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] PATH_INFO: `/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1/groups/f83ebc2986d34eb6b93e7d9c655e6188/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.884853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:25.885010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] RBAC: Authorizing `identity:create_grant(project_id=5460a823bf554ac5a99fd5d73c21c6d1, group_id=f83ebc2986d34eb6b93e7d9c655e6188, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:25.887290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:25.909709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 202/418] 10.222.0.25 () {58 vars in 1442 bytes} [Fri Jun 17 11:25:25 2022] PUT /identity/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1/groups/f83ebc2986d34eb6b93e7d9c655e6188/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:25.918591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:25.921099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:25.921649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:25.922146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:25.923217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:25.923825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:25.925583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:25.936980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 217/419] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:25.943376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:26.082672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb145888-7308-4aa2-806c-f1cff3e0c1ed None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:26.119881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:26.132512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:26.133445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:26.134026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:26.134499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:26.206942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:26.208737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:26.333287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 203/420] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:26.339032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:26.467841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2492c2-fa67-46d7-83b5-66a665be17a3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:26.511741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:26.513026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:26.513528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:26.514028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:26.515443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:26.517091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:26.519368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 218/421] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:26 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:26.524471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:26.552859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:26.643275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e653a-ca29-4429-b692-172cb358042f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:26.692117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937fefa-8f4a-4862-ac6d-c42927619f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:26.693050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937fefa-8f4a-4862-ac6d-c42927619f88 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:26.693340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937fefa-8f4a-4862-ac6d-c42927619f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:26.693612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937fefa-8f4a-4862-ac6d-c42927619f88 None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:26.694838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937fefa-8f4a-4862-ac6d-c42927619f88 None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:26.696335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937fefa-8f4a-4862-ac6d-c42927619f88 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:26.697979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 204/422] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:26 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:26.704049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:26.752123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:26.831905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d5ea9-aedb-4361-bd7c-c99f720a7929 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:26.868706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6b007-c779-4793-a84d-64645db45f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:26.869425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b007-c779-4793-a84d-64645db45f45 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:26.869711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b007-c779-4793-a84d-64645db45f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:26.870014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6b007-c779-4793-a84d-64645db45f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:26.870910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6b007-c779-4793-a84d-64645db45f45 None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=577d26cace0e4d3fba244021e34a6d6b) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:26.871227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6b007-c779-4793-a84d-64645db45f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:26.872687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6b007-c779-4793-a84d-64645db45f45 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:26.879141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 219/423] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:26 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=9ebaaf52f4cd402880b403c50600168e => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:27.034265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b8f6-c129-4b02-b7c5-257218ab3a8a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.034677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b8f6-c129-4b02-b7c5-257218ab3a8a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.035175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b8f6-c129-4b02-b7c5-257218ab3a8a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.037658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 205/424] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:27.043306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7879b8-40b3-48be-b744-85cddd4fea22 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.043603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7879b8-40b3-48be-b744-85cddd4fea22 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.044027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7879b8-40b3-48be-b744-85cddd4fea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.066760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0c7879b8-40b3-48be-b744-85cddd4fea22 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:27.067954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0c7879b8-40b3-48be-b744-85cddd4fea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:27.094463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e652-07aa-4cf9-bcb3-c4c8f88bf625 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.094994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e652-07aa-4cf9-bcb3-c4c8f88bf625 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.094994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79e652-07aa-4cf9-bcb3-c4c8f88bf625 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.096536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 206/425] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:27.104120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7ff25-c2c2-4ae9-88ba-fda38fc03eca None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.104806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7ff25-c2c2-4ae9-88ba-fda38fc03eca None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.105282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7ff25-c2c2-4ae9-88ba-fda38fc03eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.127062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d8d7ff25-c2c2-4ae9-88ba-fda38fc03eca None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:27.128370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d8d7ff25-c2c2-4ae9-88ba-fda38fc03eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:27.156842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 220/426] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:27.193125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7879b8-40b3-48be-b744-85cddd4fea22 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:27.229431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 207/427] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:27.261214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7ff25-c2c2-4ae9-88ba-fda38fc03eca None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:27.269795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:27.297610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:27.360100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7879b8-40b3-48be-b744-85cddd4fea22 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:27.400114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7ff25-c2c2-4ae9-88ba-fda38fc03eca None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:27.409741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:27.411308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.411789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.412884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.428079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:27.437573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:27.438497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:27.439293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:27.441379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 221/428] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:27.446563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:27.447413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:27.448430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.448898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.449435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.467327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:27.477919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:27.478796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 17 11:25:27.479609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:27.481964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 208/429] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:27.485911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:27.567043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:27.571884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272c794-11f3-4498-b6cd-46f3e84f6aab None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:27.590803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:27.621396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95e964-18d9-4bf3-805f-3b966b3e178d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:27.633061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:27.634118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.634627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.635106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.636068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:27.636586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:27.638930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:27.648138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 222/430] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:27.654078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:27.662724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:27.663403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.663685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.663961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.664769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:27.665078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:27.666494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:27.683170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 209/431] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:25:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:27.689398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:27.780810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bf534-65e2-4f5c-a3e8-d93f64c341d9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:27.799287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fdc4b-4c81-4fa5-8def-3603d99e490d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:27.819250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:27.819833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.820119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.820924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.838357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:27.839066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:27.839365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:27.839645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:27.857972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:27.858854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:27.903179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:27.905159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:27.926975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:27.928852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.007736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 223/432] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:25:28.013347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.038677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 210/433] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:25:28.043849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.091862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:28.093059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:28.148056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700e2eb-ad6e-44cc-abf0-f6fc51ede5f6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.150461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e694b2-fcb6-46aa-a6ef-24a940bd7d03 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.188260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.189132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.189562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.189953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] PATH_INFO: `/v3/projects/577d26cace0e4d3fba244021e34a6d6b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.191363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] RBAC: Authorizing `identity:get_project(project_id=577d26cace0e4d3fba244021e34a6d6b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:28.193088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.193624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.194708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.195163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.195328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 224/434] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/projects/577d26cace0e4d3fba244021e34a6d6b => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:28.195753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.196972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:28.201218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.201345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.203874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 211/435] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:28.207671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.313292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e69ec-f9b8-4049-83aa-21bf66868a16 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.324803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3251f-9f3b-4374-bbd8-42241ba06c53 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.340900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:28.341501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:28.350256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913d720-7bf8-4cc6-a20f-cc6af3a9518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.350679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913d720-7bf8-4cc6-a20f-cc6af3a9518c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.350963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913d720-7bf8-4cc6-a20f-cc6af3a9518c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.351284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913d720-7bf8-4cc6-a20f-cc6af3a9518c None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.352429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913d720-7bf8-4cc6-a20f-cc6af3a9518c None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:28.353876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913d720-7bf8-4cc6-a20f-cc6af3a9518c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.360365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 225/436] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:28.361597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883eeb-072e-45b7-960a-64095afb956a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.361906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883eeb-072e-45b7-960a-64095afb956a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.362234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883eeb-072e-45b7-960a-64095afb956a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.366057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 226/437] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:28.366057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97883eeb-072e-45b7-960a-64095afb956a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.367682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53353b8-a812-4746-8c37-037275ed78b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.368558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53353b8-a812-4746-8c37-037275ed78b9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.368923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53353b8-a812-4746-8c37-037275ed78b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.369392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53353b8-a812-4746-8c37-037275ed78b9 None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.371471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53353b8-a812-4746-8c37-037275ed78b9 None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:28.373458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53353b8-a812-4746-8c37-037275ed78b9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.375112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 212/438] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:28.376683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc23a41-874a-442c-b058-510a2e4911b3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.376982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc23a41-874a-442c-b058-510a2e4911b3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.377268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc23a41-874a-442c-b058-510a2e4911b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.394834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6dc23a41-874a-442c-b058-510a2e4911b3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:28.396030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6dc23a41-874a-442c-b058-510a2e4911b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:28.468558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 213/439] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:28.470722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc23a41-874a-442c-b058-510a2e4911b3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.477043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97883eeb-072e-45b7-960a-64095afb956a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.513678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.514743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.514743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.514903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.515806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=577d26cace0e4d3fba244021e34a6d6b) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:28.516124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:28.517545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.523980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 227/440] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=577d26cace0e4d3fba244021e34a6d6b&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:28.531448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.593630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc23a41-874a-442c-b058-510a2e4911b3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.610492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:28.635664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:28.656624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c99cfa-2e03-4115-9d3a-6dee445099ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.657756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c99cfa-2e03-4115-9d3a-6dee445099ec None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.658303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c99cfa-2e03-4115-9d3a-6dee445099ec None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.658784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c99cfa-2e03-4115-9d3a-6dee445099ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.660631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c99cfa-2e03-4115-9d3a-6dee445099ec None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=9ebaaf52f4cd402880b403c50600168e, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:28.661257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c99cfa-2e03-4115-9d3a-6dee445099ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:28.663905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c99cfa-2e03-4115-9d3a-6dee445099ec None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.674598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 214/441] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=9ebaaf52f4cd402880b403c50600168e => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:28.694757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058d6c9-79b1-42f0-8f66-f81c1e47dc0e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.706812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40decff7-2b81-4631-9fe0-8da47a1316f7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.707123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40decff7-2b81-4631-9fe0-8da47a1316f7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.707368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40decff7-2b81-4631-9fe0-8da47a1316f7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.708944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 215/442] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:28.714417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b24c1-2607-49aa-8b29-f9fd19a03ed1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.714727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b24c1-2607-49aa-8b29-f9fd19a03ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.714983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b24c1-2607-49aa-8b29-f9fd19a03ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.736529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6d1b24c1-2607-49aa-8b29-f9fd19a03ed1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:28.737904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6d1b24c1-2607-49aa-8b29-f9fd19a03ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:28.740605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.741453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.741737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.742044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.756854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:28.765638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:28.766612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:28.767417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:28.770745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 228/443] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:28.776978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.829270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 216/444] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:28.853556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b24c1-2607-49aa-8b29-f9fd19a03ed1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.902444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745dfbbe-76d4-4437-8905-62427ebcc29c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.930301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:28.948114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:28.948793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:28.949076 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:28.949351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:28.950176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:28.950493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:28.951862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:28.961154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 229/445] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:28.967597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:28.985887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b24c1-2607-49aa-8b29-f9fd19a03ed1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:28.996624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.031365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.032210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.032556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.032779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.044849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:29.055154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:29.056071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:29.056869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 217/446] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:29.062097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.100633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca8f09-8111-495a-9b72-7c49f8946195 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.145359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.146078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.146379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.146650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.178916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc61a1-2eef-48f8-b35e-8faa4619ab31 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.197522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.213607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.214315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.214595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.214879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.216198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:29.216553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.217983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.220130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.222737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.227377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 218/447] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:29.232950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.260976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.328550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 230/448] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:25:29.333891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.338436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e18186-d0b0-4c46-ab09-1c1f8e9dfb9f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.373370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.374057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.374345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.374623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.414888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.441056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5921ba-2892-4fbe-b5ea-81c8d21be568 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.442012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.443656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.472731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.480044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.480905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.481187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.481463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] PATH_INFO: `/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.482673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] RBAC: Authorizing `identity:get_project(project_id=c2e3bce33fec47b5b23c70bedccc7dc9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.484094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.485979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 231/449] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/projects/c2e3bce33fec47b5b23c70bedccc7dc9 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:29.491191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.546322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 219/450] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:25:29.551834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.589677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d5dde-cbfc-4b16-aa5c-b558001c9c85 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.626221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.627077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.627357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.627628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] PATH_INFO: `/v3/groups/18a9de2b95654f0187c3efefff2ef7a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.628898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] RBAC: Authorizing `identity:get_group(group_id=18a9de2b95654f0187c3efefff2ef7a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.630369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.636998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 232/451] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/groups/18a9de2b95654f0187c3efefff2ef7a7 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:29.638477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.648191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2e2f-b5d5-4d35-a7c8-73d6ae1f03cc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.656818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.674371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.682009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.682866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.683152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.683423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] PATH_INFO: `/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.685125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] RBAC: Authorizing `identity:get_project(project_id=5460a823bf554ac5a99fd5d73c21c6d1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.686568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.688240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 220/452] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/projects/5460a823bf554ac5a99fd5d73c21c6d1 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:29.693526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.739938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450a05a-7b08-4aff-bed1-754f17541f0d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.774836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.775486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.775772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.776048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.776888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] RBAC: Adding query filter params (group.id=18a9de2b95654f0187c3efefff2ef7a7, role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:29.777210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.778663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.784587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 233/453] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/role_assignments?group.id=18a9de2b95654f0187c3efefff2ef7a7&scope.project.id=c2e3bce33fec47b5b23c70bedccc7dc9&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:29.792060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a47fa-c9cd-45bf-a9a4-67bf49ce70dd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.839303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f55ec4-8aad-49bd-9299-59c328dbd6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.840204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55ec4-8aad-49bd-9299-59c328dbd6fd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.840488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55ec4-8aad-49bd-9299-59c328dbd6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.840763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55ec4-8aad-49bd-9299-59c328dbd6fd None admin] PATH_INFO: `/v3/groups/f83ebc2986d34eb6b93e7d9c655e6188` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.842076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f55ec4-8aad-49bd-9299-59c328dbd6fd None admin] RBAC: Authorizing `identity:get_group(group_id=f83ebc2986d34eb6b93e7d9c655e6188)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.843583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f55ec4-8aad-49bd-9299-59c328dbd6fd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:29.845390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 221/454] 10.222.0.25 () {56 vars in 1180 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/groups/f83ebc2986d34eb6b93e7d9c655e6188 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:29.852199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:29.938475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:29.958453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530176c-3488-427c-acb2-07fad6ed1f21 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:29.992672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8c822-0db0-4b68-bd7f-3a0e2fa83482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:29.993328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8c822-0db0-4b68-bd7f-3a0e2fa83482 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:29.993615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8c822-0db0-4b68-bd7f-3a0e2fa83482 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:29.993919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8c822-0db0-4b68-bd7f-3a0e2fa83482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:29.994760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8c822-0db0-4b68-bd7f-3a0e2fa83482 None admin] RBAC: Adding query filter params (group.id=f83ebc2986d34eb6b93e7d9c655e6188, role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=5460a823bf554ac5a99fd5d73c21c6d1) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:29.995086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8c822-0db0-4b68-bd7f-3a0e2fa83482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:29.996505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8c822-0db0-4b68-bd7f-3a0e2fa83482 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:30.003088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 234/455] 10.222.0.25 () {56 vars in 1376 bytes} [Fri Jun 17 11:25:29 2022] GET /identity/v3/role_assignments?group.id=f83ebc2986d34eb6b93e7d9c655e6188&scope.project.id=5460a823bf554ac5a99fd5d73c21c6d1&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:32.492978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0bcb1-984e-4bd2-9371-520123ea403f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:32.493513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0bcb1-984e-4bd2-9371-520123ea403f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:32.493513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0bcb1-984e-4bd2-9371-520123ea403f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:32.495161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 222/456] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:32.500636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c2acc-b177-46cb-a39c-a02720c736b8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:32.501404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c2acc-b177-46cb-a39c-a02720c736b8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:32.501883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c2acc-b177-46cb-a39c-a02720c736b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:32.522905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-739c2acc-b177-46cb-a39c-a02720c736b8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:32.524107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-739c2acc-b177-46cb-a39c-a02720c736b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:32.612332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 235/457] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:32.657750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0bcb1-984e-4bd2-9371-520123ea403f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:32.744424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:32.805565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0bcb1-984e-4bd2-9371-520123ea403f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:32.843685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c6eca-1352-452e-9bd7-c6ac6a0980ca None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:32.843970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c6eca-1352-452e-9bd7-c6ac6a0980ca None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:32.844237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c6eca-1352-452e-9bd7-c6ac6a0980ca None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:32.848232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 236/458] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:32.854134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31489b04-aa11-4461-85b7-f8e0ac034967 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:32.854421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31489b04-aa11-4461-85b7-f8e0ac034967 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:32.854700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31489b04-aa11-4461-85b7-f8e0ac034967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:32.855587 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:32.856431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:32.856720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:32.856996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:32.874918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:32.876385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:32.878740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-31489b04-aa11-4461-85b7-f8e0ac034967 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:32.879939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-31489b04-aa11-4461-85b7-f8e0ac034967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:32.907631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:32.908691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:32.909903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:32.909903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:32.909903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:32.909903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:32.914010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 223/459] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:32.914875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.012175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 237/460] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:33.020576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d701-6c86-418d-afe0-2af8fd314f1f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.020865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d701-6c86-418d-afe0-2af8fd314f1f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.021138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d701-6c86-418d-afe0-2af8fd314f1f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.022753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 238/461] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:33.027706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df054-4ed1-4d48-8fcb-d4644efc9be2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.028030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df054-4ed1-4d48-8fcb-d4644efc9be2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.028313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df054-4ed1-4d48-8fcb-d4644efc9be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.048864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d9a2-3ef1-4783-b402-d096f28d2d1a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.049034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:33.063121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c14df054-4ed1-4d48-8fcb-d4644efc9be2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:33.064353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c14df054-4ed1-4d48-8fcb-d4644efc9be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:33.070220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:33.093998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.094991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.095278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.095557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.096602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:33.096944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.099143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.116077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 224/462] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:33.117055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.158758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 239/463] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:33.162992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14df054-4ed1-4d48-8fcb-d4644efc9be2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.237157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa5e4-c53b-4400-a4b1-8e1a37054e7c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.273629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea24936-0d4c-43ea-b170-4ef48792640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.274380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24936-0d4c-43ea-b170-4ef48792640c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.274667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24936-0d4c-43ea-b170-4ef48792640c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.274939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24936-0d4c-43ea-b170-4ef48792640c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.276313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14df054-4ed1-4d48-8fcb-d4644efc9be2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.286530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea24936-0d4c-43ea-b170-4ef48792640c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.288552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea24936-0d4c-43ea-b170-4ef48792640c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7ea24936-0d4c-43ea-b170-4ef48792640c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:33.306380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:33.307242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:33.308157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:33.308157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:33.308157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:33.308157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:33.312631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 225/464] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:33.313810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c8c2-fd58-4a23-b307-36c2921e76cb None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.314150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c8c2-fd58-4a23-b307-36c2921e76cb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.314429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c8c2-fd58-4a23-b307-36c2921e76cb None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.315854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 226/465] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:33.316486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c8c2-fd58-4a23-b307-36c2921e76cb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.320914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.321723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.322039 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.322318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.323031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.324475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.334931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:33.352249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:33.355218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 240/466] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:33 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:33.356924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.459017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c8c2-fd58-4a23-b307-36c2921e76cb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.482857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00afad-275b-4285-917b-811f84ff7da9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.499045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe59d41-ae44-44dc-85ac-d9facd0d7173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.499888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe59d41-ae44-44dc-85ac-d9facd0d7173 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.500592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe59d41-ae44-44dc-85ac-d9facd0d7173 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.500592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe59d41-ae44-44dc-85ac-d9facd0d7173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.513653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe59d41-ae44-44dc-85ac-d9facd0d7173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.515301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe59d41-ae44-44dc-85ac-d9facd0d7173 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.523459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.524783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.525229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.525513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.526282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:33.526598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.528500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4fe59d41-ae44-44dc-85ac-d9facd0d7173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:33.534638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:33.535462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:33.536288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:33.536288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:33.536288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:33.536288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:33.537792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 227/467] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:33.539160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642fed7-9bab-4ed8-a2db-15983898c1e8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.539160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642fed7-9bab-4ed8-a2db-15983898c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.540862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642fed7-9bab-4ed8-a2db-15983898c1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.547634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 241/468] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:33.548829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.559512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9642fed7-9bab-4ed8-a2db-15983898c1e8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:33.560710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9642fed7-9bab-4ed8-a2db-15983898c1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:33.608459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:33.658792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 228/469] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:33.659947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642fed7-9bab-4ed8-a2db-15983898c1e8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.668750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65965df1-8f9e-41d3-a1bb-b1edafb18a9c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.674086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:33.711593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.712509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.713066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.713490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.714401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:33.714978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.717295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.730335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 242/470] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:33.735675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.776324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642fed7-9bab-4ed8-a2db-15983898c1e8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.812939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.813627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.813954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.814237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.814950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.816697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.844838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 229/471] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:33 2022] POST /identity/v3/users => generated 248 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:33.846107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.850162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61992d9b-135c-439a-9a0b-f3e80b48310d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:33.869210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:33.894418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:33.895224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:33.895510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:33.895787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:33.909708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:33.911207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:33.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:33.930840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:33.931685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:33.931685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:33.931685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:33.931685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:33.931685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 243/472] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:33.939254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:33.942104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:34.010136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1021b-369d-424a-af38-20f0065797bd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:34.085709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788f6e-1b61-4fcf-b5d8-88c6ed210ecb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:34.086970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:34.088782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:34.089152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:34.089450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:34.090183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:34.091692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:34.123230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 230/473] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:33 2022] POST /identity/v3/users => generated 248 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:34.126794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fce5ce-da49-48f2-96af-9718dc8b54a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:34.127441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fce5ce-da49-48f2-96af-9718dc8b54a1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:34.127722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fce5ce-da49-48f2-96af-9718dc8b54a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:34.127994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fce5ce-da49-48f2-96af-9718dc8b54a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:34.128724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fce5ce-da49-48f2-96af-9718dc8b54a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:34.129028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fce5ce-da49-48f2-96af-9718dc8b54a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:34.131254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fce5ce-da49-48f2-96af-9718dc8b54a1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:34.146781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 244/474] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:34.152417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:34.276525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f57e5-6f5f-464a-ac42-8b6ee577f73f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:34.302933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:34.320600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd4ce5-3477-42b5-ba42-2bc8e085eb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:34.321311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd4ce5-3477-42b5-ba42-2bc8e085eb7e None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:34.321600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd4ce5-3477-42b5-ba42-2bc8e085eb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:34.321911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd4ce5-3477-42b5-ba42-2bc8e085eb7e None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:34.322693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd4ce5-3477-42b5-ba42-2bc8e085eb7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:34.324176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd4ce5-3477-42b5-ba42-2bc8e085eb7e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:34.353941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 231/475] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:34 2022] POST /identity/v3/users => generated 249 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:34.395238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de255f-bca7-4f25-b3b5-4ff393ceeba0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:34.395538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de255f-bca7-4f25-b3b5-4ff393ceeba0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:34.395802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de255f-bca7-4f25-b3b5-4ff393ceeba0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:34.397584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 245/476] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:34.403663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73293e-8d38-4c76-a21f-a2ccfe4dfbc3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:34.403965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73293e-8d38-4c76-a21f-a2ccfe4dfbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:34.404248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73293e-8d38-4c76-a21f-a2ccfe4dfbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:34.427605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be73293e-8d38-4c76-a21f-a2ccfe4dfbc3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:34.429654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be73293e-8d38-4c76-a21f-a2ccfe4dfbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:34.513089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 232/477] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:34.538678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de255f-bca7-4f25-b3b5-4ff393ceeba0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:34.573186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:34.680180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de255f-bca7-4f25-b3b5-4ff393ceeba0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:34.719852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:34.720534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:34.720821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:34.721098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:34.721885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:34.723262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:34.739386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.sql.core [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 11:25:34.739386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 11:25:34.739386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [parameters: {'id': '20bb7d0dbe084e5795688a52a6bcbd6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 11:25:34.739386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80103) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 17 11:25:34.756648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return ref.to_dict() Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:25:34.757714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application yield resource Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise self.value Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application session.commit() Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 11:25:34.758843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.flush() Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application compat.raise_( Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 11:25:34.759971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:25:34.761032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:25:34.766155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [parameters: {'id': '20bb7d0dbe084e5795688a52a6bcbd6e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:34.767124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 17 11:25:34.768107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:34.770577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 246/478] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Jun 17 11:25:34 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:34.776660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73293e-8d38-4c76-a21f-a2ccfe4dfbc3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:34.934865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73293e-8d38-4c76-a21f-a2ccfe4dfbc3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:34.945041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:34.972391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276bb50-2ab8-42b0-bac0-febc6a21debb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:34.973213 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276bb50-2ab8-42b0-bac0-febc6a21debb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:34.973496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276bb50-2ab8-42b0-bac0-febc6a21debb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:34.973769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276bb50-2ab8-42b0-bac0-febc6a21debb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:34.988632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276bb50-2ab8-42b0-bac0-febc6a21debb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f276bb50-2ab8-42b0-bac0-febc6a21debb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:34.996776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:34.997657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:34.998465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 233/479] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:25:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:35.003835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:35.138232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:35.143392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b38e5-0fec-492e-be6d-9730c62c785b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:35.183072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff6ca7-ba68-4d2c-bb82-292e1f32e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:35.184112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6ca7-ba68-4d2c-bb82-292e1f32e1ae None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:35.184590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6ca7-ba68-4d2c-bb82-292e1f32e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:35.185135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff6ca7-ba68-4d2c-bb82-292e1f32e1ae None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:35.186098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6ca7-ba68-4d2c-bb82-292e1f32e1ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:35.186620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6ca7-ba68-4d2c-bb82-292e1f32e1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:35.188193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff6ca7-ba68-4d2c-bb82-292e1f32e1ae None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:35.205950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 247/480] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:35.870415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4661f93-4433-4ade-aa84-ff4c40e3640f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:35.870918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4661f93-4433-4ade-aa84-ff4c40e3640f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:35.870995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4661f93-4433-4ade-aa84-ff4c40e3640f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:35.873117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 234/481] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:35.878726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033d83-0fcc-44c6-8a67-27ccea4b9701 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:35.879136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033d83-0fcc-44c6-8a67-27ccea4b9701 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:35.879326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b033d83-0fcc-44c6-8a67-27ccea4b9701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:35.901878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3b033d83-0fcc-44c6-8a67-27ccea4b9701 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:35.903024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3b033d83-0fcc-44c6-8a67-27ccea4b9701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:36.003243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 248/482] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:36.044507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4661f93-4433-4ade-aa84-ff4c40e3640f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:36.190599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4661f93-4433-4ade-aa84-ff4c40e3640f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:36.237568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:36.238410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:36.238699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:36.238978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:36.252585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:36.260391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:36.261264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:36.262825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 235/483] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:36.268066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b033d83-0fcc-44c6-8a67-27ccea4b9701 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:36.415033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:36.427513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b033d83-0fcc-44c6-8a67-27ccea4b9701 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:36.464177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:36.464836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:36.465272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:36.465564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:36.466357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:36.466669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:36.468052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:36.477000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 249/484] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:36.482919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:36.549036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:36.670293 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c0a5e-6a03-4097-a6f9-62d1598d5e8e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:36.717454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:36.718219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:36.718506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:36.718781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:36.737896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:36.739419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:36.759050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:36.759930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:36.761545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:36.761545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:36.761545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:36.761545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:36.761545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 236/485] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:36.767796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:36.888522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea2a3c-7b6d-4ee0-bc2e-2e365b4b1e7e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:36.945731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94351833-8e33-4157-846b-125adf9d40cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:36.946456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94351833-8e33-4157-846b-125adf9d40cd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:36.946744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94351833-8e33-4157-846b-125adf9d40cd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:36.947023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94351833-8e33-4157-846b-125adf9d40cd None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:36.947774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94351833-8e33-4157-846b-125adf9d40cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:36.948088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94351833-8e33-4157-846b-125adf9d40cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:36.949536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94351833-8e33-4157-846b-125adf9d40cd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:36.969059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 250/486] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:36.974922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.029217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9ac95-6246-48d1-9919-f9beeca97eae None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.029868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9ac95-6246-48d1-9919-f9beeca97eae None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.030374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9ac95-6246-48d1-9919-f9beeca97eae None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.033190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 251/487] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:37.037806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2ea1a-5724-455f-a02f-bd17a5363b53 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.038368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2ea1a-5724-455f-a02f-bd17a5363b53 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.038841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2ea1a-5724-455f-a02f-bd17a5363b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.044620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.059545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0ac2ea1a-5724-455f-a02f-bd17a5363b53 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:37.061068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0ac2ea1a-5724-455f-a02f-bd17a5363b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:37.085468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.108344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede405-7e8a-46d6-aacd-e6791ece5701 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.135115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 252/488] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 17 11:25:37.136613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff79bd-3b20-4ab1-bc49-0075578a1680 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.136907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff79bd-3b20-4ab1-bc49-0075578a1680 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.137212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff79bd-3b20-4ab1-bc49-0075578a1680 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.138632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 253/489] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:37.139575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbb2d6-ae90-4374-9a99-866af7320227 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.139836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbb2d6-ae90-4374-9a99-866af7320227 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.140114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbb2d6-ae90-4374-9a99-866af7320227 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.141435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 254/490] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:37.149884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2108b-a913-4c81-9e20-2d77430fc91c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.149884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2108b-a913-4c81-9e20-2d77430fc91c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.149884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2108b-a913-4c81-9e20-2d77430fc91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.154057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31498f-8a71-44b4-abec-263d4946e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.155836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31498f-8a71-44b4-abec-263d4946e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.156130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31498f-8a71-44b4-abec-263d4946e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.156416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31498f-8a71-44b4-abec-263d4946e5c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.165669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-feb2108b-a913-4c81-9e20-2d77430fc91c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:37.165831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31498f-8a71-44b4-abec-263d4946e5c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:37.166882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-feb2108b-a913-4c81-9e20-2d77430fc91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:37.167254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31498f-8a71-44b4-abec-263d4946e5c1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9b31498f-8a71-44b4-abec-263d4946e5c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:37.187657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:37.188556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:37.189440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:37.190489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 237/491] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:25:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:37.191156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922abb2d-620e-4e95-8304-79ac8d705e79 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.191442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922abb2d-620e-4e95-8304-79ac8d705e79 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.191724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922abb2d-620e-4e95-8304-79ac8d705e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.208801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-922abb2d-620e-4e95-8304-79ac8d705e79 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:37.210014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-922abb2d-620e-4e95-8304-79ac8d705e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:37.249946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 255/492] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:37.249946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2108b-a913-4c81-9e20-2d77430fc91c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.286798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 238/493] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:37.290816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922abb2d-620e-4e95-8304-79ac8d705e79 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.315599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.361097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2108b-a913-4c81-9e20-2d77430fc91c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.375543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.398559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.399358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.399638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.400059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.409079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922abb2d-620e-4e95-8304-79ac8d705e79 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.411189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:37.419106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:37.420032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:37.420849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 256/494] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:37.422187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.452320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.453285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.453557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.454428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:37.454895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:37.456185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:37.497419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 239/495] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:37.497419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.534994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c542a21-943f-4b5c-be91-58ea2d12c1b9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.559483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.572330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.573132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.573416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.573682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.584776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:37.592916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:37.593820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:37.594936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:37.599108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 257/496] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:37.599763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.603214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c7330-ab4d-422f-a588-d12e92803432 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.633371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.645726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.646584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.646863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.647132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.659214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:37.667039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:37.667892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:37.668834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 240/497] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:37.669895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.719947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430052b3-c6da-4412-be24-5294c01d1dd0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.754726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.755398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.755678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.755953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.756727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:37.757004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:37.758404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:37.771361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 258/498] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:37.771361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:37.776868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccee1bb-191c-4f7a-abaa-f88e7019ad48 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.813777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.814385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.814670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.815355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.820892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.876329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:37.882083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13368fc8-b814-4282-907f-4f4c4c26f6f6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:37.891283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:37.892852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:37.919288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:37.920536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:37.920826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:37.921104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:37.921880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:37.922203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:37.923566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:37.936743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 259/499] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:37.936743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.005515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 241/500] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:38.006446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.048391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db731d3-745f-4db3-9680-3cb064913e13 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.055639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.083718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.084327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.084616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.084896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.085643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:38.085990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.087361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.097496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 260/501] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:38.098167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.107482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.120441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dc696-9e61-4913-92a5-d2f8caac9ce2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.154668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.155487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.155777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.156046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.166228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.167892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:38.187128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:38.188036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:38.188872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:38.188872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:38.188872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.188872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:38.188872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 242/502] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:38.189575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.211655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45a78-0d51-4531-9da1-f21be54327c2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.245255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98b136-3f97-46e4-82b0-7b5415c3b673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.246138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98b136-3f97-46e4-82b0-7b5415c3b673 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.247041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98b136-3f97-46e4-82b0-7b5415c3b673 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.247331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98b136-3f97-46e4-82b0-7b5415c3b673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.257766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98b136-3f97-46e4-82b0-7b5415c3b673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.259223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98b136-3f97-46e4-82b0-7b5415c3b673 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5f98b136-3f97-46e4-82b0-7b5415c3b673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:38.275743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:38.276786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:38.277702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:38.277702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:38.277702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.277702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:38.277702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 261/503] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:38.278518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec832000-b6f2-4b0a-90cc-58a2e1e58765 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.278801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec832000-b6f2-4b0a-90cc-58a2e1e58765 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.279074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec832000-b6f2-4b0a-90cc-58a2e1e58765 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.280458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 262/504] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:38.281273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec832000-b6f2-4b0a-90cc-58a2e1e58765 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.290445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894704a-a4f9-41ff-93e5-dc3a9ebcd838 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.303463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.325735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.326564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.326846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.327126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.335349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.337958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.340282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:38.358853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:38.359681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:38.360557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:38.360557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:38.360557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.360557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:38.360557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 243/505] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:38.360851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.385816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec832000-b6f2-4b0a-90cc-58a2e1e58765 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.419324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.420114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.420396 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.420931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.437418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.439866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:38.456810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:38.457687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:38.458551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:38.458551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:38.458551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.458551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:38.458551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 263/506] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:38.458821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.468852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dac052-9c78-484c-8e7b-5e4e7fe0e19e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.504089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc6e8e-aa35-435c-b29b-cc826c9d3e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.504761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc6e8e-aa35-435c-b29b-cc826c9d3e7d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.505047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc6e8e-aa35-435c-b29b-cc826c9d3e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.505326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc6e8e-aa35-435c-b29b-cc826c9d3e7d None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.506075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc6e8e-aa35-435c-b29b-cc826c9d3e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:38.506389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc6e8e-aa35-435c-b29b-cc826c9d3e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.507764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc6e8e-aa35-435c-b29b-cc826c9d3e7d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.518793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 244/507] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:38.519993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640063f-9473-4a6c-965c-5db3a4acff9b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.520290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640063f-9473-4a6c-965c-5db3a4acff9b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.520559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640063f-9473-4a6c-965c-5db3a4acff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.529819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.536715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b640063f-9473-4a6c-965c-5db3a4acff9b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:38.538305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b640063f-9473-4a6c-965c-5db3a4acff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:38.561062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538cc6e-c7dc-43ef-a969-9c730e11dac9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.584289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.594700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.595304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.595585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.595858 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.596569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:38.596879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.598261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.604505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 245/508] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:38.605315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640063f-9473-4a6c-965c-5db3a4acff9b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.619149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 264/509] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:38.619950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.706287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640063f-9473-4a6c-965c-5db3a4acff9b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.723566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d57af2-a791-4dab-b418-539baeded0d2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.740717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.741468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.741760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.742068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.742792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:38.743102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.744484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.752827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.756117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 246/510] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:38.756924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.758598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.759421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.759706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.759982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.761503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:38.761815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.763244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.775008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 265/511] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:38.775783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.802837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:38.860069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab038283-6171-4a55-b756-b372d5e0524d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.875430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da037a-090f-4cb4-91cf-577da5a409ec None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:38.892857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.893650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.893971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.894247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.902348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.904365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.909100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:38.909909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:38.910190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:38.910461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:38.919809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:38.920709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:38.921597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 247/512] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:38.922168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:38.922563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.922644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:38.942563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:38.943440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:38.949291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:38.949291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:38.949291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:38.949291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:38.949291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 266/513] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:38.949291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:38.989680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.025305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73919014-2630-4281-a5c7-9007543ed48d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.026864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.044636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd94d2d-b629-475c-a215-e4e170459148 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.059109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.059897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.060501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.067716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.069139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.077933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.078718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.079011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.079290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:39.083268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:39.084114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:39.085006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 248/514] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:39.085764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.087035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.088483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:39.104064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:39.105062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:39.105984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 267/515] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:39.106907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.184665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c2b48-8e23-4703-ba1e-8082c077fa3b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.209379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918caf58-2879-461e-a7b0-26584883fffd None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.211842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.217858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.218662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.218936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.219206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.226546 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.227980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:39.242109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:39.243098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:39.244017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 249/516] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:39.244561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.248387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.249062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.249346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.249619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.250774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:39.251086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.252533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.271615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.289807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 268/517] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:39.290655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.360605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b7609-4bdb-4384-98aa-6cefc7224460 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.392967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b5942-9084-4e6a-bfd0-1cbc5e2520a5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.399726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.400401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.400682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.400954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.401719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:39.402056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.403420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.415505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 250/518] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:39.416308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.426732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.427401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.427681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.427973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.428786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:39.429094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.430541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.448427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 269/519] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:39.449166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.471651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.498412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.516613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4faadc-231e-4064-8b0c-65ada9bd3d18 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.549332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.550035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.550321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.550573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c05324-234d-4167-bd1f-189a9d5c30f2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.550675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.551432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:39.551740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.553181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.571395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 251/520] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:39.572198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.585830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.586526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.586813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.587092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.587938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:39.588259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.589677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.607165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 270/521] 10.222.0.25 () {56 vars in 1139 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:39.607949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.685739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42a217-00ee-4009-846e-21fba44f8106 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.686376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.707762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfefce-16e3-4dc7-83af-e9f482f93930 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.718913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.719565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.719850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.720128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.732102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.744698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.745465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.745645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.745947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.747253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.748674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.769364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 271/522] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:39 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:39.770180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.802733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:39.804262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:39.890827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d31462-3fa7-4f61-8a7d-a9819a3192a3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:39.923125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 252/523] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:39.923125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:39.938249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:39.939562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:39.939562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:39.939562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:39.942475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:39.972401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.024164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.026135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.035911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdfa72-7bfb-425c-bc60-6d26298c8a13 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.071882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.072547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.072837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.073121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.073984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=804a35678c91438eae36f61c1de8c579) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:40.074328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.075786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.081271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 253/524] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/role_assignments?user.id=804a35678c91438eae36f61c1de8c579&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:40.082084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.158231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 272/525] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:40.159254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.197693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.205790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acd8c1-cec8-4a4d-9dd4-37591e12a962 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.214280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.247441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.248467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.248904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.249316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.273467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446110c-0c00-462a-9cb6-c2ae4ab44a97 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.319855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.320669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.320968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.321496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.332363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.334116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.335339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.337065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:40.363688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:40.364576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:40.365405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:40.365405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:40.365405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:40.365405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:40.365405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 273/526] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:40.366141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.446316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 254/527] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:40.447551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.456573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.471539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.491245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a46fd7-a518-4009-a091-49b1ff5aafc1 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.535211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.536313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.536872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.537713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.550076 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.551986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.554003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e0273-9718-4d69-8f36-7d908be37b71 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:40.572902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:40.573928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:40.574818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:40.574818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:40.574818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:40.574818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:40.581396 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 274/528] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:40.582636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.589866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.590578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.590860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.591135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.591858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:40.592164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.593529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.606344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 255/529] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:40.607264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.679818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.700539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0a1be-121b-4443-bf31-6cb56041f1b9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.716835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370e56f-9d2e-42eb-a101-52752426e587 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.737882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.738958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.739458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.739931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.747565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.753748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.754518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.754996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.755489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.755857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.756589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:40.756895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.757255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.758546 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.772342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 256/530] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:40.773245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:40.778237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:40.779072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:40.779939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:40.779939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:40.779939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:40.779939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:40.782127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 275/531] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:40.783200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.879229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb7b86-9b7f-4ea7-8958-20e5b6fd80bf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.890057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdf7a3-11be-4251-b2e9-8cce0a4c149b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:40.908563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:40.920507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.921264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.921822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.922101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.929688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.931465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.933826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:40.934959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:40.935447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:40.935937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:40.943505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:40.945649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:40.947307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:40.948109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:40.949373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 257/532] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:40.949831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:40.961115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:40.962012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:40.962872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 276/533] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:40.963765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:41.020224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:41.054001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f2436a-42fb-4af1-8193-171262be3a29 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.078835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34bc3b-f70d-47d2-8a9b-69921d9d301c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.087622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.088287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.088566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.088842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.089558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:41.090417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.091807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.104617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 258/534] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:41.105454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:41.124788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.125570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.125897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.126184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.128150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:41.128150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.128874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.150144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 277/535] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:41.151004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:41.163398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:41.248713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c5b6f-6372-4269-9df5-6d1028ca7c51 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.281319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce388f5-7362-459e-9f8b-e4e66086743f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.297215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:41.301375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.303397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.304109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.304534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.305754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:41.306638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.308234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.327396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 259/536] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:41.328792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.328959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:41.329964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.330659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.331166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.340543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.342365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:41.361163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:41.362604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:41.363635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:41.366702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 278/537] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:41.367767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:41.495910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d10a3a-ecfc-45dd-be43-8b165562d0a4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.496112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d79f8-bbbf-49f0-b336-68038709dec0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.508093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:41.548908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.549586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.549910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.550198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.551166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=17154b0972cb4534a5c379babcb9d2f7) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:41.551488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.553015 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.553867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.555413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.555937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.556439 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.557785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=86f3b6ba74fb49a280735bb8c867d26e) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:41.558354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.560588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.561804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 279/538] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:41 2022] GET /identity/v3/role_assignments?user.id=17154b0972cb4534a5c379babcb9d2f7&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:41.562732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:41.574035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 260/539] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:41 2022] GET /identity/v3/role_assignments?user.id=86f3b6ba74fb49a280735bb8c867d26e&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:41.610762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:41.702882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aeca0-67c0-44d6-b3ec-6c1184537751 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.746771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34517e01-a6af-4f51-8e24-a2a7d12ffc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.747545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34517e01-a6af-4f51-8e24-a2a7d12ffc5a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.747834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34517e01-a6af-4f51-8e24-a2a7d12ffc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.748116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34517e01-a6af-4f51-8e24-a2a7d12ffc5a None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.748952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34517e01-a6af-4f51-8e24-a2a7d12ffc5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:41.749270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34517e01-a6af-4f51-8e24-a2a7d12ffc5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.750802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34517e01-a6af-4f51-8e24-a2a7d12ffc5a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.765897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 280/540] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:41.773539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:41.912795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7bea-7c1d-483c-855d-abdd5a933b80 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:41.960606 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81dbe-28ac-4657-a7f1-c3e3ffdb7993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:41.961676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81dbe-28ac-4657-a7f1-c3e3ffdb7993 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:41.962180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81dbe-28ac-4657-a7f1-c3e3ffdb7993 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:41.962657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81dbe-28ac-4657-a7f1-c3e3ffdb7993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:41.963713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf81dbe-28ac-4657-a7f1-c3e3ffdb7993 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=b3b35d6d2cd848f4841352fbb58f2053) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:41.964240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf81dbe-28ac-4657-a7f1-c3e3ffdb7993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:41.965948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf81dbe-28ac-4657-a7f1-c3e3ffdb7993 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:41.974309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 261/541] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:41 2022] GET /identity/v3/role_assignments?user.id=b3b35d6d2cd848f4841352fbb58f2053&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:42.186852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060b67-c090-4b6c-a028-44dfc3efc2f9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:42.187830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060b67-c090-4b6c-a028-44dfc3efc2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:42.188348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060b67-c090-4b6c-a028-44dfc3efc2f9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:42.190204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 281/542] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:42.195498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e0ae-03db-46c2-a3c5-7ebeed7b59fb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:42.196401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e0ae-03db-46c2-a3c5-7ebeed7b59fb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:42.196903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e0ae-03db-46c2-a3c5-7ebeed7b59fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:42.217867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-abd3e0ae-03db-46c2-a3c5-7ebeed7b59fb None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:42.217867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-abd3e0ae-03db-46c2-a3c5-7ebeed7b59fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:42.252222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:42.299274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 262/543] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:42.339495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4060b67-c090-4b6c-a028-44dfc3efc2f9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:42.464364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:42.479694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4060b67-c090-4b6c-a028-44dfc3efc2f9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:42.531787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b887e-e2d3-4ced-ba69-f927cbf73ed3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:42.532074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b887e-e2d3-4ced-ba69-f927cbf73ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:42.532352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b887e-e2d3-4ced-ba69-f927cbf73ed3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:42.532579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:42.533552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:42.534002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:42.534444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:42.538002 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 263/544] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:42.539391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3112d-569a-4d8b-aafe-1527916753ed None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:42.539671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3112d-569a-4d8b-aafe-1527916753ed None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:42.539947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3112d-569a-4d8b-aafe-1527916753ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:42.559498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:42.575312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:42.576471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:42.577374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 282/545] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:42.582233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0cd3112d-569a-4d8b-aafe-1527916753ed None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:42.583595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0cd3112d-569a-4d8b-aafe-1527916753ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:42.584949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:42.659362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 264/546] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:42.681833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3112d-569a-4d8b-aafe-1527916753ed None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:42.692602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0de697-4191-4bde-b37e-52b6001c16d4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:42.732734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:42.733341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:42.733628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:42.733932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:42.734698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:42.735011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:42.736392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:42.755344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 283/547] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:42.759941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:42.798525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:42.829717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:42.859790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3112d-569a-4d8b-aafe-1527916753ed None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:42.886440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09b794-1523-4038-8c49-47ae0c600ffb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:42.923526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:42.924240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:42.924997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:42.924997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:42.937823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:42.939566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:42.960370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:42.961259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:42.962120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:42.962120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:42.962120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:42.962120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:42.966067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 284/548] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:42.967540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:43.002561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:43.003394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:43.003681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:43.003959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:43.044663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:43.062482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:43.070257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:43.072206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:43.074670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:43.076273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb50f1-dec9-494e-9047-e3562e33dcbe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:43.081859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 265/549] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:43.082758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:43.113869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:43.114627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:43.114916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:43.115203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:43.116311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:43.116630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:43.118057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:43.135364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 285/550] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:43.136575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:43.268653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6eefbd-440f-4a76-86d2-a7a93133dff5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:43.300113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37614d12-3c1b-44b3-a451-f2ac214d0800 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:43.322126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:43.338758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:43.339729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:43.341243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:43.341622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:43.357476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:43.358965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:43.369641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:43.370355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:43.370641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:43.370920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:43.371723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:43.372036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:43.373408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:43.399165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:43.401389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:43.404259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 286/551] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:43.408617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 266/552] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:43.412550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:43.413567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:43.442954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:43.578160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f6ff-0d17-49af-bca8-ccf8d3c5eaa7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:43.596567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4acf3-9165-4fe3-a07d-92da67127ee1 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:43.641486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:43.645863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:43.645863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:43.645863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:43.645863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:43.645863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:43.645863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:43.655442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:43.656522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:43.657163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:43.657636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:43.669866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 287/553] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Jun 17 11:25:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:43.671147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:43.678932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:43.680605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:43.710943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:43.711898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:43.712748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:43.712748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:43.712748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:43.712748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:43.715504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 267/554] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:43.721028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:43.721381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:43.778346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcd7d7-c41f-46db-96eb-75f4982972e2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:43.817130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:43.818217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:43.818738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:43.819209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:43.883775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:43.894132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:43.895995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:43.959850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:43.977390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780038a-af3f-4f82-b625-1d1f0a2fe02d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.032979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 288/555] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:25:44.036305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.052898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.053928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.054437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.055274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.056197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:44.056701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:44.058311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:44.080690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 268/556] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:44.087278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.172612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fc7c8-2aa1-4688-baac-d024ed0168b5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.216354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fd731-863f-4d91-af6e-30364deeddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.217161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fd731-863f-4d91-af6e-30364deeddc0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.217446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fd731-863f-4d91-af6e-30364deeddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.217723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fd731-863f-4d91-af6e-30364deeddc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.239568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fd731-863f-4d91-af6e-30364deeddc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:44.241046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fd731-863f-4d91-af6e-30364deeddc0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:44.251638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-7d1fd731-863f-4d91-af6e-30364deeddc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:44.263437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:44.264781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:44.265809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:44.265809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:44.265809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:44.265809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:44.265809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 289/557] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:44.266695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78459-596a-4c38-9a79-d832f3f24ab0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.266976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78459-596a-4c38-9a79-d832f3f24ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.267345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f78459-596a-4c38-9a79-d832f3f24ab0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.268767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 290/558] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:44.273260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f78459-596a-4c38-9a79-d832f3f24ab0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.292679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940984c-b8da-4b26-a219-3087bfdba022 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.330314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:44.341673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159983b-150c-4bea-8168-5df2247575c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.342878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159983b-150c-4bea-8168-5df2247575c3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.343221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159983b-150c-4bea-8168-5df2247575c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.343503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159983b-150c-4bea-8168-5df2247575c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.354342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159983b-150c-4bea-8168-5df2247575c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:44.356718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159983b-150c-4bea-8168-5df2247575c3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9159983b-150c-4bea-8168-5df2247575c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:44.378745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:44.379787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:44.380805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:44.384966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 269/559] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:44.387284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768de551-757e-4c65-8134-930433d2d07f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.387576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768de551-757e-4c65-8134-930433d2d07f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.387855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768de551-757e-4c65-8134-930433d2d07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.406572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f78459-596a-4c38-9a79-d832f3f24ab0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.410872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-768de551-757e-4c65-8134-930433d2d07f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:44.412425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-768de551-757e-4c65-8134-930433d2d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:44.448938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.449564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.449872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.450155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.450889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:44.451202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:44.452568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:44.467616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 291/560] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:44.468822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.497380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 270/561] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:44.498740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c10b2-226f-40e5-86cb-07e778e15de0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.499036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c10b2-226f-40e5-86cb-07e778e15de0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.499318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c10b2-226f-40e5-86cb-07e778e15de0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.500589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 271/562] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:44.501288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c10b2-226f-40e5-86cb-07e778e15de0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.533744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:44.577029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143866f8-5db4-4a0c-b1b4-a3af34578411 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.603392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c10b2-226f-40e5-86cb-07e778e15de0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.610918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18269c-3b55-44a1-ad34-816340d97559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.611591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18269c-3b55-44a1-ad34-816340d97559 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.611876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18269c-3b55-44a1-ad34-816340d97559 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.612155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18269c-3b55-44a1-ad34-816340d97559 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.612994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18269c-3b55-44a1-ad34-816340d97559 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:44.613304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18269c-3b55-44a1-ad34-816340d97559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:44.614805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18269c-3b55-44a1-ad34-816340d97559 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:44.622440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:44.635728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 292/563] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:44.637136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec181-e16f-4d97-a1f9-3e6064ba5edc None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.637429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec181-e16f-4d97-a1f9-3e6064ba5edc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.637730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec181-e16f-4d97-a1f9-3e6064ba5edc None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.638856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.639656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.640181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.640181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.643316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 293/564] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:44.644228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ff749-de5e-4876-a745-1e008af9dc8c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.644518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ff749-de5e-4876-a745-1e008af9dc8c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.644792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ff749-de5e-4876-a745-1e008af9dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.647375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:44.648826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:44.662616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-aa1ff749-de5e-4876-a745-1e008af9dc8c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:44.663386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:44.664361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:44.665239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:44.665652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-aa1ff749-de5e-4876-a745-1e008af9dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:44.665721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 272/565] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:44.665721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.734434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 294/566] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:44.735275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ff749-de5e-4876-a745-1e008af9dc8c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.770712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c4376-447c-4d4d-9ddd-4d6c51be67b8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.780768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:44.805069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f802bd8-efcc-48fe-a1be-313adc64e839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.805885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f802bd8-efcc-48fe-a1be-313adc64e839 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.806170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f802bd8-efcc-48fe-a1be-313adc64e839 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.806444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f802bd8-efcc-48fe-a1be-313adc64e839 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.816368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f802bd8-efcc-48fe-a1be-313adc64e839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6f802bd8-efcc-48fe-a1be-313adc64e839 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:44.823951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:44.824905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:44.825720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 273/567] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:44.826948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a2f69-f8c6-47d5-85d8-f1af92d81815 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.826948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a2f69-f8c6-47d5-85d8-f1af92d81815 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.826948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a2f69-f8c6-47d5-85d8-f1af92d81815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.838463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ff749-de5e-4876-a745-1e008af9dc8c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:44.843278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f37a2f69-f8c6-47d5-85d8-f1af92d81815 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:44.844461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f37a2f69-f8c6-47d5-85d8-f1af92d81815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:44.874663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:44.876035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:44.876459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:44.876748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:44.882340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:44.887793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:44.889266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:44.907788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:44.908662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:44.909522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:44.909522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:44.909522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:44.909522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:44.909522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 295/568] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:44.909800 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:44.929260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 274/569] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:44.930131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a2f69-f8c6-47d5-85d8-f1af92d81815 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.006636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.010999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940b38-9af4-4f04-9306-0b4b4a243956 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.045459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a2f69-f8c6-47d5-85d8-f1af92d81815 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.046044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.046707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.046998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.047275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.048123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:45.048444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.049881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.061464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 296/570] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:45.062279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.079214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.080018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.080300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.080576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.091674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:45.099489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:45.100406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:45.101222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 275/571] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:45.101933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.129208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.162848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d523f8-05bd-45f5-9713-c56b99c64c49 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.195341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.196011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.196299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.196577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.197322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:45.197635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.199568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.201266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9570159-08a8-45c4-9dad-83ddd0793c0c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.207471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 297/572] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:45.208267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.229708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.240533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.241204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.241486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.241759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.242512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:45.242817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.244216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.256598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 276/573] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:45.256598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.312433 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a5747-d946-41ac-ab20-bcd768d95317 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.346725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.346931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.347522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.347846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.348204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.358637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:45.359720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81a40-cd4c-4aac-9158-7690408b8a8a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:45.366346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:45.367271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:45.368088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 298/574] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:45.368765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.397137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.397800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.398112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.398387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.399284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=fd3119ad342840ddb0258e124be55597) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:45.399673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.401034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.406710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 277/575] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/role_assignments?user.id=fd3119ad342840ddb0258e124be55597&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:45.407503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.466128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.488790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc201c6-a4ed-4d78-a474-459ddff8965d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.528391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.529086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.529371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.529654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.529884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696ee1b-f7c8-41d5-82c2-97b68d656f92 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.530460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:45.530774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.532196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.546201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 299/576] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:45.546201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.576608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.577442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.577723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.578031 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.590972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.592433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:45.609731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:45.610816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:45.611724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:45.611724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:45.611724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:45.611724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:45.611724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 278/577] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:45.613884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.618002 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.662162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead49ae-bf47-4944-a955-313b7d1a2968 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.699033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.699951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.700110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.700567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.708098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.709542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.724171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc1c9b-6f65-4ac2-894e-d6eafd4df233 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:45.724329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:45.725141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 300/578] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:45.731236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.733036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.768757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.769444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.769731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.770041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.770797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:45.771105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.772475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.785190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 279/579] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:45.785190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.848799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad9743-fce7-403a-a419-664b4796e5be None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.850693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:45.885652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.886505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.886769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.887045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.892728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c4267-75eb-4376-af57-325529a8b5dc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:45.897235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.898715 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:45.916186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:45.917282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:45.918748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:45.918748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:45.918748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:45.918748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:45.918748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 301/580] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:45.919037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.935204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911044-1f0b-4260-978b-36479241646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:45.936100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911044-1f0b-4260-978b-36479241646d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:45.936630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911044-1f0b-4260-978b-36479241646d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:45.937058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b911044-1f0b-4260-978b-36479241646d None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:45.937990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b911044-1f0b-4260-978b-36479241646d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:45.938446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b911044-1f0b-4260-978b-36479241646d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:45.939956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b911044-1f0b-4260-978b-36479241646d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:45.956770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 280/581] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:45.956770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911044-1f0b-4260-978b-36479241646d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:45.983785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.032646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa996f2-a4bc-41c6-8c33-97139b60bcc4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.066425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b911044-1f0b-4260-978b-36479241646d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.069138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.069807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.070119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.070398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.071191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:46.071506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.072920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.089369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 302/582] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:46.089369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.097439 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.108092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.108908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.109206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.109470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.120246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.121735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:46.140108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:46.141004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:46.142336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:46.142336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:46.142336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:46.142336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:46.142336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 281/583] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:46.142336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.197607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f041d4-e674-4b75-94d6-c68377849e07 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.212440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.235063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.235924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.236210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.236492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.237219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:46.237532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.238932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.249354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7858-65e2-4a72-9db1-d2d0b130f513 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.255718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 303/584] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:46.255718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.291176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.291984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.292268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.292708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.300475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.301920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:46.316455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:46.317369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 282/585] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:46.318669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.361731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.362352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006008c-d296-4ef0-a984-ea3037542634 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.400267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.401272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.401551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.401820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/804a35678c91438eae36f61c1de8c579/roles/b3c5094c32f446d88c32145d8b8ae3cd` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.417510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=804a35678c91438eae36f61c1de8c579, role_id=b3c5094c32f446d88c32145d8b8ae3cd)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.419001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.427045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6fc13-40f9-441e-8a9a-761d216771b6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.441881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 304/586] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:46 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/804a35678c91438eae36f61c1de8c579/roles/b3c5094c32f446d88c32145d8b8ae3cd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:46.441881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.468906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.469795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.470115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.470390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.471152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:46.471457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.473279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.475290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.486851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 283/587] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:46.487887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.587354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fee9a-1e4b-4e60-8acb-f0d9ebeefb94 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.614389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.623793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1baf9a4-eb67-4fa8-b8ec-a4cdf973c7c6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.634829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.635869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.636156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.636431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.646156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.648098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.664202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.665262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.665752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.666284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.667409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:46.668493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:46.668598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:46.669455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:46.670474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 305/588] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:46.671044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.671115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.695893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 284/589] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:46.696744 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.798040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84514fd2-040e-4c7f-ae74-e3e3330d5d76 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.805908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f79b0-a5a0-4373-bc15-2a4ef96e22cc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:46.826158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.835940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.837034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.837652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.838180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.841865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:46.842598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:46.842885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:46.843169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:46.843995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:46.844304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.845812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.849282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:46.850993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:46.864845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 285/590] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:46.865625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:46.868169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:46.869096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:25:46.870084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:46.872751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 306/591] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:46.873827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:46.882177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:46.981304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2758d22-c9d6-4b25-9492-7dbabfb18d49 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.001801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155229-9fb4-4072-9bd4-eca2f29d32ee None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.019160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.020443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.020728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.021273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.035291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.037741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.045544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.046394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.047217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.047500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.059077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:47.060183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:47.061047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:47.061960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:47.061960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:47.061960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:47.061960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:47.062235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.062311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 286/592] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:47.062311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.069937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:47.088673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:47.091306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:47.093414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:47.093414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:47.093414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:47.093414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:47.101860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 307/593] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:47.101860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.135041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.172309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb8b15-77d4-4644-af1b-96bacdd10fdd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.203180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec07ad-82f6-41cc-819c-f1b67417360f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.208343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.209027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.209309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.209581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.210458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:47.210763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.212185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.231524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 287/594] 10.222.0.25 () {56 vars in 1139 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:47.232437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.244782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.245509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.245820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.246129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.246923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:47.247288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.248721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.262069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 308/595] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:47.262922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.322932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.341862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fc61d-bc8c-4d68-81c2-411f9bbc4ff4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.366981 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.372349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f0530-e0ea-4862-8a9b-4b4ee530d7c6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.379374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.380042 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.380323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.380601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.381335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:47.381642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.383045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.395579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 288/596] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:47.396416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.415224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.416164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.416453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.416833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.432007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.433630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:47.455990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:47.458368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:47.461939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:47.461939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:47.461939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:47.461939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:47.462795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 309/597] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:47.469870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.500310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fca78-937d-45d9-bd44-8e44bf504cdc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.539543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.540361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.540641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.540916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.548295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.549741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:47.564839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:47.566982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:47.569078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:47.574004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 289/598] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:47.574004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.581172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c7ace-dc0b-40ee-aec8-6b4a88ec2db6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.590815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.623030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.626832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.627989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.628272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.628545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.641235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.642748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:47.661275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:47.662455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:47.663316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:47.667924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 310/599] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:47.668851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.679389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f7b88-5dea-41dd-94a2-cb93ab39f2ff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.719829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e559b-ce58-47f9-ac72-6403d3ce7ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.720615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e559b-ce58-47f9-ac72-6403d3ce7ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.720901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e559b-ce58-47f9-ac72-6403d3ce7ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.721184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e559b-ce58-47f9-ac72-6403d3ce7ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.721938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e559b-ce58-47f9-ac72-6403d3ce7ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:47.722250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e559b-ce58-47f9-ac72-6403d3ce7ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.723634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e559b-ce58-47f9-ac72-6403d3ce7ab9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.735533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 290/600] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:47.736821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd58638-d42f-4d41-94c3-d8bd06ebb749 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.737114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd58638-d42f-4d41-94c3-d8bd06ebb749 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.737393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd58638-d42f-4d41-94c3-d8bd06ebb749 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.738800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 291/601] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:47.743277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd58638-d42f-4d41-94c3-d8bd06ebb749 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.785756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be6002-4dc4-4b37-ac4e-ba8bc0c15875 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.820817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.821486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.821768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.822094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.822892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:47.823204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.825184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.837894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 311/602] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:47.838136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:47.862838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd58638-d42f-4d41-94c3-d8bd06ebb749 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.865866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.869667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:47.905676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fc6cc-c8b1-48e3-bd56-c85032b31a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.906681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fc6cc-c8b1-48e3-bd56-c85032b31a6b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.906968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fc6cc-c8b1-48e3-bd56-c85032b31a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.907378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fc6cc-c8b1-48e3-bd56-c85032b31a6b None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.908167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fc6cc-c8b1-48e3-bd56-c85032b31a6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:47.908481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fc6cc-c8b1-48e3-bd56-c85032b31a6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:47.909919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fc6cc-c8b1-48e3-bd56-c85032b31a6b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:47.927723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 292/603] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:47.927723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa7bc5-6ab7-44e7-b950-da1db1952a7b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.927723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa7bc5-6ab7-44e7-b950-da1db1952a7b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.927723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa7bc5-6ab7-44e7-b950-da1db1952a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:47.944185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6aaa7bc5-6ab7-44e7-b950-da1db1952a7b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:47.945399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6aaa7bc5-6ab7-44e7-b950-da1db1952a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:47.955582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50519-162f-4dd6-baa2-222462bd9a8f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:47.991981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:47.992853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:47.993136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:47.993416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.001292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:48.002742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:48.016932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:48.017784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:48.018704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 312/604] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:48.019188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.025891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 293/605] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:48.025891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa7bc5-6ab7-44e7-b950-da1db1952a7b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.122939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:48.135347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b24110-c8fb-4ec9-9af3-baeeb2aaa802 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:48.136089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:48.144441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa7bc5-6ab7-44e7-b950-da1db1952a7b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:48.172180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:48.173074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.173367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.173647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/86f3b6ba74fb49a280735bb8c867d26e/roles/b3c5094c32f446d88c32145d8b8ae3cd` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.187758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:48.188593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.188885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.189160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/17154b0972cb4534a5c379babcb9d2f7/roles/b3c5094c32f446d88c32145d8b8ae3cd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.192662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=86f3b6ba74fb49a280735bb8c867d26e, role_id=b3c5094c32f446d88c32145d8b8ae3cd)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:48.194144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:48.206237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=17154b0972cb4534a5c379babcb9d2f7, role_id=b3c5094c32f446d88c32145d8b8ae3cd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:48.207690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:48.208146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 313/606] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:48 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/86f3b6ba74fb49a280735bb8c867d26e/roles/b3c5094c32f446d88c32145d8b8ae3cd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:48.208751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.249251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 294/607] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:48 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/17154b0972cb4534a5c379babcb9d2f7/roles/b3c5094c32f446d88c32145d8b8ae3cd => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:25:48.251041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.358975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7573d-e555-44e0-8179-2e066ef53f07 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:48.380196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040245ca-07fd-475c-88ed-0124a3b76d2e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:48.413325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:48.414055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.415371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.416430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.417287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:48.417635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:48.422176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:48.435807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:48.438435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:48.439256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.439548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.439883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.453473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 314/608] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:48.457980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.461618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:48.479305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:48.482199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:48.484174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 295/609] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:48.490025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.496983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:48.591486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543059f3-5b44-4e8a-9e08-a44ab39304e3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:48.640999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c466b9f-acb6-4108-ab87-ac4762b33b89 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:48.643828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:48.644935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.645366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.645781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/b3b35d6d2cd848f4841352fbb58f2053/roles/b3c5094c32f446d88c32145d8b8ae3cd` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.674144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=b3b35d6d2cd848f4841352fbb58f2053, role_id=b3c5094c32f446d88c32145d8b8ae3cd)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:48.675850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:48.702038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 315/610] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:48 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/b3b35d6d2cd848f4841352fbb58f2053/roles/b3c5094c32f446d88c32145d8b8ae3cd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:25:48.709974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5dd573-c544-480d-bf61-bb319b30ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:48.711136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dd573-c544-480d-bf61-bb319b30ef7a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.711677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dd573-c544-480d-bf61-bb319b30ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.712229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dd573-c544-480d-bf61-bb319b30ef7a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.713258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dd573-c544-480d-bf61-bb319b30ef7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:48.713818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dd573-c544-480d-bf61-bb319b30ef7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:48.716228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5dd573-c544-480d-bf61-bb319b30ef7a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:48.726888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 296/611] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:48.732350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.777707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c725f-d93f-4d5d-89c6-4d82f98f8f23 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.778430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c725f-d93f-4d5d-89c6-4d82f98f8f23 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.778904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c725f-d93f-4d5d-89c6-4d82f98f8f23 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.780637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 297/612] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:48.786690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c71d4-f084-433b-b32f-390c8f13baec None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.787562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c71d4-f084-433b-b32f-390c8f13baec None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.788073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c71d4-f084-433b-b32f-390c8f13baec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.817265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-237c71d4-f084-433b-b32f-390c8f13baec None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:48.820376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-237c71d4-f084-433b-b32f-390c8f13baec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:48.866235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:48.880135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b34c-fc9a-418b-80a1-eae4628f8706 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:48.919333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:48.920281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:48.920667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:48.921035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:48.928224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:48.932010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:48.934317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 298/613] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:48.935587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:48.954832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:48.955760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:48.956659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:48.956659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:48.956659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:48.956659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:48.959347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 316/614] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:48.960621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c71d4-f084-433b-b32f-390c8f13baec None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:48.962174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:49.082839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:49.088668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9d63-66e1-4303-9466-815acbfc31ae None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:49.105601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c71d4-f084-433b-b32f-390c8f13baec None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:49.126365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:49.127069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:49.127359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:49.127640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:49.128537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:49.128920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:49.130634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:49.153013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 317/615] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:49.163040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:49.165791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:49.166597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:49.166884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:49.167163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:49.194786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:49.211317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:49.212280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:49.213741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 299/616] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:49.218298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:49.283261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc7056-bc56-4634-8de9-cc47b5e7d787 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:49.320581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:49.321748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:49.322193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:49.322632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:49.331918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:49.333691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:49.350033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:49.351169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:25:49.352297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:49.354679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 318/617] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:49.361876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:49.366312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:49.410689 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:49.448949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bcc4a-3a5d-4336-82da-e6e908c221a4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:49.506400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:49.507088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:49.507373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:49.507648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:49.508408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:49.508715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:49.510128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:49.528765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 300/618] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:49.535525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:49.575109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb34d3-e379-4d20-b8ec-e209912d40c5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:49.655814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:49.657883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:49.658346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:49.658807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:49.660111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:49.660602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:49.662587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:49.684911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 319/619] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Jun 17 11:25:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:49.691161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:49.753683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cf82c-aff9-4a12-8850-42c1416c846a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:49.787196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:49.820309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:49.821410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:49.821887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:49.822333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:49.830172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:49.844046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:49.846247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:49.874797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:49.875831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:49.876789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:49.876789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:49.876789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:49.876789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:49.879218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fa569-96de-46aa-b1b9-7bcb76757302 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:49.882526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 301/620] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:49.888311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:49.928352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:49.929040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:49.929329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:49.929600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:49.939890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:49.941564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:49.976361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:49.977299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:49.978230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:49.978230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:49.978230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:49.978230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:49.986191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 320/621] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:49.991584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:50.039239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee8ddc-60bb-4aff-999f-381c6c18cfa7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.122569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.123299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.123591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.123873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.124622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:50.125143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:50.126397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:50.146253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693aa04-1716-49ec-bf40-a64a4703c8e7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.161648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 302/622] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:50.182911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:50.187488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:50.203894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.204579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.205583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.206108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.206861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:50.207177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:50.210940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:50.230801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 321/623] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:50.236400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:50.237530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:50.408756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c333be-4b1d-4a67-b719-1a8a8a37b8df None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.421656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475c3bc-e78a-4a59-a6cc-94303376bf15 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.479661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.480516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.480807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.481080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.485332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.486159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.486441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.486712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.494408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:50.495896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:50.500864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:50.502525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:50.519033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:50.520212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:50.521887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 303/624] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:25:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:50.525512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:50.526362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:50.529221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:50.531358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 322/625] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:50.536224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:50.626236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:50.702633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:50.713057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a550433-c50a-4bfa-a0b9-d4e164bb4e11 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.721761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ac44c-947a-4ee4-b1dd-d047346dce29 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.752989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.753676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.754004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.754286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.755141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:50.755564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:50.757009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:50.776613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 304/626] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:25:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:50.787253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:50.791515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.792190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.792503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.792796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.793594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:50.793936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:50.795371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:50.807679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 323/627] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:50.813894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:50.911577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b37c32-88ff-4d1b-8664-ba2be3a960c3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.932857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d2388-fd3d-4458-9ce2-528ce7e848e3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:50.945242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:50.955219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.955804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.956089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.956365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.971121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e150663-6b02-48dc-83bf-eb3e063febf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:50.972041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e150663-6b02-48dc-83bf-eb3e063febf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:50.972327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e150663-6b02-48dc-83bf-eb3e063febf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:50.972606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e150663-6b02-48dc-83bf-eb3e063febf1 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/fd3119ad342840ddb0258e124be55597/roles/b3c5094c32f446d88c32145d8b8ae3cd` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:50.976387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:50.989211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e150663-6b02-48dc-83bf-eb3e063febf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=fd3119ad342840ddb0258e124be55597, role_id=b3c5094c32f446d88c32145d8b8ae3cd)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:50.990718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e150663-6b02-48dc-83bf-eb3e063febf1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:51.006628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 324/628] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:25:50 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/fd3119ad342840ddb0258e124be55597/roles/b3c5094c32f446d88c32145d8b8ae3cd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:25:51.011691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b5e57-8e0d-4515-a8d6-13e4a4cc7302 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.012066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b5e57-8e0d-4515-a8d6-13e4a4cc7302 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.012358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b5e57-8e0d-4515-a8d6-13e4a4cc7302 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.014041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 325/629] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:51.019547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b1e8-1b17-4227-82f5-3c18dede0d1d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.019851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b1e8-1b17-4227-82f5-3c18dede0d1d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.020151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957b1e8-1b17-4227-82f5-3c18dede0d1d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.021522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 326/630] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:51.026634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4e8e-5393-4f23-b956-f5a3f7c21a59 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.026979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4e8e-5393-4f23-b956-f5a3f7c21a59 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.027236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4e8e-5393-4f23-b956-f5a3f7c21a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.039168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:51.041027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:51.049178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-12ca4e8e-5393-4f23-b956-f5a3f7c21a59 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:51.050533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-12ca4e8e-5393-4f23-b956-f5a3f7c21a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:51.130486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 327/631] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:51.132704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9439c14-3335-4a3c-a413-07ee5813e029 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.132704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9439c14-3335-4a3c-a413-07ee5813e029 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.132704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9439c14-3335-4a3c-a413-07ee5813e029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.153901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b9439c14-3335-4a3c-a413-07ee5813e029 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:51.153901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b9439c14-3335-4a3c-a413-07ee5813e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:51.179875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 305/632] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:51.185487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.239366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 328/633] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:51.239963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9439c14-3335-4a3c-a413-07ee5813e029 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.244622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:51.264542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:51.308923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f082e-5da6-40ef-adb9-514b69d79835 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.350258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.351080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.351366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.351644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.361673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9439c14-3335-4a3c-a413-07ee5813e029 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.363544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:51.371382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:51.372304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:51.373537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 306/634] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:51.374290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.405211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.406158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.406361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.406587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.418320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:51.419740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:51.437303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:51.438266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:51.443839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:51.443839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:51.443839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:51.443839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:51.443839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 329/635] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:51.443839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.493884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c0b48-0b56-45d5-a935-89d1e857236b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.505764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:51.514000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:51.534962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05497759-b218-4bc6-b626-057e8e8a0b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.536253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05497759-b218-4bc6-b626-057e8e8a0b7f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.536253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05497759-b218-4bc6-b626-057e8e8a0b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.536433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05497759-b218-4bc6-b626-057e8e8a0b7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.547708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05497759-b218-4bc6-b626-057e8e8a0b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:51.550002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5113e1-743f-4fad-87d8-f48f4b26abd6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-05497759-b218-4bc6-b626-057e8e8a0b7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:51.555499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:51.556435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:51.557933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 307/636] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:51.558679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdc179-498d-4b7a-93e2-cfd7fc12ea91 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.558679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdc179-498d-4b7a-93e2-cfd7fc12ea91 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.558810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdc179-498d-4b7a-93e2-cfd7fc12ea91 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.560256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 308/637] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:51.564641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdc179-498d-4b7a-93e2-cfd7fc12ea91 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.593434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.594200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.594489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.594763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.595510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:51.595820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:51.597175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:51.610444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 330/638] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:51.610444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.684660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdc179-498d-4b7a-93e2-cfd7fc12ea91 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.718477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30e76-7877-46d4-876a-8c611368406a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.724702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549914-c04e-4216-9305-d104e96a1210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.725395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549914-c04e-4216-9305-d104e96a1210 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.725661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549914-c04e-4216-9305-d104e96a1210 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.726388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549914-c04e-4216-9305-d104e96a1210 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.727122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549914-c04e-4216-9305-d104e96a1210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:51.727436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549914-c04e-4216-9305-d104e96a1210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:51.729519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549914-c04e-4216-9305-d104e96a1210 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:51.749325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 309/639] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:51.749325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe571ab-22e5-40ea-a209-646a4949a663 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.749325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe571ab-22e5-40ea-a209-646a4949a663 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.749325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe571ab-22e5-40ea-a209-646a4949a663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.753243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.753938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.754223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.754499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.755250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:51.755559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:51.756919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:51.758377 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:51.766410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 331/640] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:51.767213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.771805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1fe571ab-22e5-40ea-a209-646a4949a663 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:51.772995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1fe571ab-22e5-40ea-a209-646a4949a663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:51.798175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:51.845279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 310/641] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:51.846440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe571ab-22e5-40ea-a209-646a4949a663 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.886035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196e49e-9601-405b-a8ae-14d8eb4acc33 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.921469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.922603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.923077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.923531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.934390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:51.936044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:51.953944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe571ab-22e5-40ea-a209-646a4949a663 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:51.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:51.955296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:51.956172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:51.956172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:51.956172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:51.956172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:51.958404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 332/642] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:51.959383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:51.990766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:51.991565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:51.991847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:51.992120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:51.998709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.000215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.010733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:52.015961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:52.016894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:52.017795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 311/643] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:52.018329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.042392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.068267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e0d36-a712-4511-b8e4-eb9d5507b7e7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.102560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.103639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.104120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.104478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.115826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.117246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.128834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e5896-82f4-44ba-8a36-a8a122240037 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:52.134308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:52.135191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:52.140782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:52.140782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:52.140782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:52.140782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:52.140782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 333/644] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:52.140782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.172268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.173087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.173376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.173658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.186136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:52.193965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:52.194935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:52.195749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:52.200078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 312/645] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:52.200803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.255214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.258692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee03bb-279f-4ba9-93ae-cd6ec3a0ef3d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.292921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.298174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.299104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.299390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.299834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.300860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:52.301177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.302566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.320220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 334/646] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:52.320387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba58b38-ae6e-410f-baea-d1feeff86d65 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.320470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.358460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.359121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.359402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.359673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.360455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:52.360761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.362207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.380146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 313/647] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:52.380992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.422263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc611f5-0245-44a3-ad5b-e85db84695c7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.457862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.458655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.458945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.459222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.459973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:52.460290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.461649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.473343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 335/648] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:52.474616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.490596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc8bed-21d1-415f-8c3c-4bb77dc45c14 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.500924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.530147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.530830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.531199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.531542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.532464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:52.532719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.534095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.540190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.550006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 314/649] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:52.550946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.586976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12874a-e245-47e7-9a04-81d3b491a6dc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.623453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.624254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.624537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.624810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.633173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.634379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:52.648540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:52.649467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:52.650437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 336/650] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:52.651037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.669539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10b41a-0159-4867-a226-0da098821fe5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.705820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.706511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.706800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.707083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.707916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=804a35678c91438eae36f61c1de8c579) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:52.708235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.709695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.715828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 315/651] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/role_assignments?user.id=804a35678c91438eae36f61c1de8c579&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:52.716621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.768266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.797512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:52.803628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee1546-0226-4d40-b395-5a3426c82a34 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.844520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f98c8-d037-4e60-84c2-f7be77da34a7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:52.857864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.858999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.859761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.860759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.873166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.875999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.882746 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:52.883565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:52.883915 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:52.884996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:52.897186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:52.898383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 17 11:25:52.899525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:52.901326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:52.902062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 337/652] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:52.902873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:52.903066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:52.922283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:52.923283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:52.924192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:52.924192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:52.924192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:52.924192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:52.924192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 316/653] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:52.924464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.029893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c09aee-c8a2-4939-85d2-39929a1e58fa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.057304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146feb54-f725-4354-a1c7-7cebe893d3ea None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.066216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.066897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.067179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.067450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.068300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:53.068609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.070126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.078185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.080315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.090744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 338/654] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:53.091651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.103924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.105513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.106039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.106523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.108087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:53.108987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.111253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.137879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 317/655] 10.222.0.25 () {56 vars in 1137 bytes} [Fri Jun 17 11:25:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:53.137879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.215914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace2ee9-2a14-4f16-af22-7faa672966fe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.256121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ab054-c245-4a83-a497-39edaf470bb1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.259348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.260190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.260645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.261069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.262588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:53.263063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.264752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.277762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 339/656] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:53.278938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.297372 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeab22b-acaf-4bcf-9e2a-3ee97d79226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.298087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeab22b-acaf-4bcf-9e2a-3ee97d79226b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.298378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeab22b-acaf-4bcf-9e2a-3ee97d79226b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.298658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeab22b-acaf-4bcf-9e2a-3ee97d79226b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.341882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.346934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.373860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeab22b-acaf-4bcf-9e2a-3ee97d79226b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.375541 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeab22b-acaf-4bcf-9e2a-3ee97d79226b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.385120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963dda6-1ec5-42ff-988d-803cff7facfb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.419712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.420366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.420653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.420935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.488718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 318/657] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:53.490410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddbad8-8fa6-43e1-9de3-8351301f54de None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.490868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddbad8-8fa6-43e1-9de3-8351301f54de None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.491294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddbad8-8fa6-43e1-9de3-8351301f54de None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.493014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 319/658] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:53.493877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddbad8-8fa6-43e1-9de3-8351301f54de None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.499481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.500950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.571222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.588130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.616125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddbad8-8fa6-43e1-9de3-8351301f54de None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.621115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 340/659] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:53.626837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.658851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c5ea-8d0d-494f-b00d-f649c221e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.659656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c5ea-8d0d-494f-b00d-f649c221e43a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.659940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c5ea-8d0d-494f-b00d-f649c221e43a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.660215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c5ea-8d0d-494f-b00d-f649c221e43a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.667735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c5ea-8d0d-494f-b00d-f649c221e43a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.669180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c5ea-8d0d-494f-b00d-f649c221e43a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-31a8c5ea-8d0d-494f-b00d-f649c221e43a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:53.684055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:53.684997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:53.685957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 320/660] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:53.686590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0649a-c6df-4b14-87c7-f54f799d2877 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.687003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0649a-c6df-4b14-87c7-f54f799d2877 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.687003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0649a-c6df-4b14-87c7-f54f799d2877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.705149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e0e0649a-c6df-4b14-87c7-f54f799d2877 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:53.706365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e0e0649a-c6df-4b14-87c7-f54f799d2877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:53.742251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd823cf-99dd-4e2e-93d9-34469db999c5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.777124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 321/661] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:53.778268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0649a-c6df-4b14-87c7-f54f799d2877 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.779917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.780779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.781133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.781490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.801827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.803617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:53.824201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:53.825102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:53.826004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:53.826004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:53.826004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:53.826004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:53.826004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 341/662] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:53.826316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.838697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.855528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:53.897355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0649a-c6df-4b14-87c7-f54f799d2877 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.934046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836672-b117-45e9-b4e9-31cdd06f3e89 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:53.936488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.937297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.937584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.937880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.948305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.949873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:53.968559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:53.969524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:53.970909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:53.970909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:53.970909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:53.970909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:53.970909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 322/663] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:53.970909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:53.977470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:53.978184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:53.978467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:53.978739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:53.979584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:53.980427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:53.981874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.000494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 342/664] 10.222.0.25 () {56 vars in 1139 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:54.001319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.074094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.082645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956aebe-213b-4126-9d16-d3551b01690d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.099525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.112796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3ac8c-c6aa-47a2-b63e-e3b223dfdb7a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.119590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.120545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.121327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.121599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.132502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:54.141278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:54.142265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:54.143163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 323/665] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:25:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:54.143847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.153103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.153710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.154023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.154297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.155027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:54.155335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.156760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.168921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 343/666] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:54.173955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.250373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b1aa4-32a5-461c-9fbc-e61c4156b64a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.272255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a585fd-d456-49fd-bded-adcf9077413c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.285622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.286994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.287450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.287898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.288896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:54.289499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.290938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.302351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.311559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 324/667] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:54.312513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.320256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.320988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.321339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.321683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.351243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.394311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.395768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.427022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871ac7-c061-442d-ac43-f59fd8b7e4af None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.461796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.462575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.462928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.463209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.463956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:54.464800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.466466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.475572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 325/668] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:54.476551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.507712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 344/669] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:54.507712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.534580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.581797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.585951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755f29c-ede5-4fd7-80b8-48c197d7ee27 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.616370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d86a1-fc1b-4e13-b51a-7efc86a8e909 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.627459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.628389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.628672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.631648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.640265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.642170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.655869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.656662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.656948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.657223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:54.658874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:54.659875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:54.660866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 326/670] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:54.662198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.670874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.672330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:54.686237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:54.687079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:54.687978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 345/671] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:54.688447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.772556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd453ef-dc5b-43a0-9106-1b6e387dc32d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.775955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.788545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7f4bf-4aa4-4cab-9dd6-daf6bab13b89 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.809329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.810163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.810448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.810723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.822103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.822462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.823265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.824088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.824193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.824371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:54.835432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:54.836871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:54.842227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:54.843063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:54.843949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:54.843949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:54.843949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:54.843949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:54.843949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 327/672] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:54.844656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.857212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:54.859213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:54.860112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:54.861005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:54.861005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:54.861005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:54.861005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:54.861005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 346/673] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:54.861301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:54.962146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870da1c-191e-45c4-ae32-61eb876a713b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.971770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87be8-270a-4dcb-b2d9-39bf085f6ced None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:54.998384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:54.999368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:54.999651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:54.999920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.000713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.001020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.002483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.013336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.014026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.014308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.014581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.015112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 328/674] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:55.015372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.015682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.015974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.017090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.026495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.031976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 347/675] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:55.032812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.087863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.122176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a979c5-403b-40df-97be-cb6e4d8e1f0b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.133356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44437f-644a-4bd8-a5a5-38ada2bc2659 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.155398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee488e-79d2-4a89-9a4c-335cda54afc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.156137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee488e-79d2-4a89-9a4c-335cda54afc5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.156420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee488e-79d2-4a89-9a4c-335cda54afc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.156777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee488e-79d2-4a89-9a4c-335cda54afc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.157494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee488e-79d2-4a89-9a4c-335cda54afc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.157801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee488e-79d2-4a89-9a4c-335cda54afc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.159201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee488e-79d2-4a89-9a4c-335cda54afc5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.167064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.167716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.168003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.168277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.169017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.169326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.170690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.171207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 329/676] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:55.172283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdafa0f-fbdd-4d3b-9914-d8fab90d64ac None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.172572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdafa0f-fbdd-4d3b-9914-d8fab90d64ac None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.172850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdafa0f-fbdd-4d3b-9914-d8fab90d64ac None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.174103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 330/677] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:55.174794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdafa0f-fbdd-4d3b-9914-d8fab90d64ac None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.190454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 348/678] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:55.190454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.244091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.278263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdafa0f-fbdd-4d3b-9914-d8fab90d64ac None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.285684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c8c9f-dc42-4525-b2b3-7053cd564386 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.305671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.312423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.313075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.313354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.313651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.314505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=17154b0972cb4534a5c379babcb9d2f7) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.314876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.316220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.320200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf78d6-d57d-45dc-8ab7-c2b11ddf825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.320844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf78d6-d57d-45dc-8ab7-c2b11ddf825f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.321131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf78d6-d57d-45dc-8ab7-c2b11ddf825f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.321409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf78d6-d57d-45dc-8ab7-c2b11ddf825f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.322064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 331/679] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/role_assignments?user.id=17154b0972cb4534a5c379babcb9d2f7&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:55.322278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf78d6-d57d-45dc-8ab7-c2b11ddf825f None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=86f3b6ba74fb49a280735bb8c867d26e) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.322598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf78d6-d57d-45dc-8ab7-c2b11ddf825f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.322826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.324594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf78d6-d57d-45dc-8ab7-c2b11ddf825f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.338085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 349/680] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/role_assignments?user.id=86f3b6ba74fb49a280735bb8c867d26e&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:55.347544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb85ba-b93b-436e-98f0-c0fdad818bec None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.348216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb85ba-b93b-436e-98f0-c0fdad818bec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.349635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb85ba-b93b-436e-98f0-c0fdad818bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.375965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-84eb85ba-b93b-436e-98f0-c0fdad818bec None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:55.378543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-84eb85ba-b93b-436e-98f0-c0fdad818bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:55.460277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330d7e3-fdb1-4a9c-87f6-07ee68602e13 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.475510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 350/681] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:55.476458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb85ba-b93b-436e-98f0-c0fdad818bec None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.520717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.522101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.522693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.523122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.535610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.537343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.554320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:55.558364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:55.559295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:25:55.560215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:55.562612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 332/682] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:55.563692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.612040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb85ba-b93b-436e-98f0-c0fdad818bec None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.634652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.660027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.660854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.661181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.661466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.671952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.674007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.688016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0eb35-9f1f-4cbb-9c35-fb06c59de09d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:55.696925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:55.697789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:55.699278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:55.700353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 351/683] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:55.701589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.737363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.739261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.739755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.740234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.754855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:25:55.765768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:55.766725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:25:55.767584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:55.775530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 333/684] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:55.775530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.837019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b223f-d8f2-4db0-a6c6-4ec496956ab8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.859660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.896218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.896926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.897192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.897465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.898493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.898809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.900378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:55.912064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12069d-63a1-45be-8fa2-518574bb63c1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:55.945863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 352/685] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:55.946080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:55.946175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:55.977863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:55.977863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:55.977863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:55.977863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:55.978328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:55.978328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:55.985868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.021866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 334/686] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:56.021866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:56.153654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67c6ab-b451-4d6f-93a1-ba9e045bb3eb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:56.193755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149d8e1-55ed-476d-9e52-73ccc384fe06 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:56.231358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:56.232027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:56.232311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:56.232589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:56.268861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:56.269563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:56.269877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:56.270147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:56.270908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:56.271218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:56.272585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.297332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:56.306451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 353/687] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:25:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:56.306451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:56.333498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:56.335402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.388807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:56.461368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 335/688] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:25:56.462741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:56.470221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb2ab1-7bed-45c3-be90-592de043f72b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:56.505032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:56.505699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:56.506017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:56.506300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:56.507151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=b3b35d6d2cd848f4841352fbb58f2053) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:56.507472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:56.508931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.521892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 354/689] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/role_assignments?user.id=b3b35d6d2cd848f4841352fbb58f2053&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:56.521892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:56.606689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:56.663895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07f3a5-6457-42c0-b8f2-eac3f47c6fba None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:56.713324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe9cee-61b3-4b90-805b-ca6833973c94 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:56.726255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:56.732205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:56.733094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:56.738175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:56.738584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:56.757556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:56.758908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:56.759885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:56.762170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.763092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:56.763587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:56.776292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:56.778204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:56.789662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:56.790598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:56.791477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:56.791477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:56.791477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:56.791477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:56.793926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 336/690] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:56.796658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:56.802132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:56.803023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:56.803921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:56.803921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:56.803921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:56.803921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:56.806348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 355/691] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:56.812361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:56.912152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e79045-d04a-43d2-8e49-fb0d7b582b1b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:56.929646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdcb93-e53e-46ae-81ad-a6b834dc1e6a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:56.939075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:56.948846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:56.950518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:56.950829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:56.951110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:56.951839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:56.952149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:56.953521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.966238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:56.966857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:56.967147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:56.967422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:56.968154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:56.968465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:56.969826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:56.969944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 337/692] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:25:56.971963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:56.983947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 356/693] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:56.989260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:57.002380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:57.080935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb4a6d-1642-4db1-8368-b09c411662fe None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:57.097487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e8de2-abb1-4b3d-82d8-7a6885315c0d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:57.116369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:57.117180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.117464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.117741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.125740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:57.127215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:57.133936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:57.134845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.135135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.135539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:25:57.143275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:57.145465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:25:57.147854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:57.149085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:57.149085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:57.149377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 338/694] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:57.153891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:57.170762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:57.172893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:57.175074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 357/695] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:25:57.180735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:57.247594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:57.325012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:57.347412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2f885-9b57-4936-9661-1bcf11271573 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:57.383828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:57.384512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.384804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.385080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.385994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:57.386309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:57.387742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:57.399678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25935462-dfb4-4ab2-b25a-d277c7d639d4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:57.412779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 339/696] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:25:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:57.414040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:57.442842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:57.443618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.443902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.444175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.445024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:57.445341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:57.447325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:57.462254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 358/697] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:57.469461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:57.625195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d60cba-7787-47e9-9dff-839d56db4cea None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:57.627011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5ddb-223e-4672-836d-5c2383dd449c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:57.634187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:57.684210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:57.695268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd39db-7385-4ab8-9b08-641c41da4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:57.695981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd39db-7385-4ab8-9b08-641c41da4933 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.696268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd39db-7385-4ab8-9b08-641c41da4933 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.696549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd39db-7385-4ab8-9b08-641c41da4933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.697455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd39db-7385-4ab8-9b08-641c41da4933 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=fd3119ad342840ddb0258e124be55597) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:57.697773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd39db-7385-4ab8-9b08-641c41da4933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:57.699943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd39db-7385-4ab8-9b08-641c41da4933 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:57.704821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb32e61-752e-41a8-b6ca-87b7e9c053fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:57.705525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb32e61-752e-41a8-b6ca-87b7e9c053fe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.705816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb32e61-752e-41a8-b6ca-87b7e9c053fe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.706120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb32e61-752e-41a8-b6ca-87b7e9c053fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.721945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 359/698] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:57 2022] GET /identity/v3/role_assignments?user.id=fd3119ad342840ddb0258e124be55597&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:57.775887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbf140-bfbf-4efd-bd6e-72d0e00dc805 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.776174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbf140-bfbf-4efd-bd6e-72d0e00dc805 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.776456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbf140-bfbf-4efd-bd6e-72d0e00dc805 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.777912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 360/699] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:25:57.778989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddff7c-a845-4a02-91bc-40908d2f999f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.779274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddff7c-a845-4a02-91bc-40908d2f999f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.779555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddff7c-a845-4a02-91bc-40908d2f999f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.780906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 361/700] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:57.786537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acdfcb-6e02-4570-9b48-28285fadc97c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.787112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acdfcb-6e02-4570-9b48-28285fadc97c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.787462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acdfcb-6e02-4570-9b48-28285fadc97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.797566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb32e61-752e-41a8-b6ca-87b7e9c053fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:57.799276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb32e61-752e-41a8-b6ca-87b7e9c053fe None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:57.815176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f1acdfcb-6e02-4570-9b48-28285fadc97c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:57.817676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f1acdfcb-6e02-4570-9b48-28285fadc97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:57.920985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 340/701] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 17 11:25:57.921422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 362/702] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:57.923351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6b801-47d5-42fd-8879-526a1f023efd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:57.923351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6b801-47d5-42fd-8879-526a1f023efd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:57.923777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6b801-47d5-42fd-8879-526a1f023efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:57.924820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acdfcb-6e02-4570-9b48-28285fadc97c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:57.932727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:57.947068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0ae6b801-47d5-42fd-8879-526a1f023efd None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:57.950104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0ae6b801-47d5-42fd-8879-526a1f023efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:58.034943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 341/703] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:58.035926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6b801-47d5-42fd-8879-526a1f023efd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.060111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acdfcb-6e02-4570-9b48-28285fadc97c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.090770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:58.107200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.107995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.108276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.108551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.121681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.123158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:58.141414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:58.142532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:58.143830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:58.143830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:58.143830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:58.143830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:58.143830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 363/704] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:58.143830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.171892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6b801-47d5-42fd-8879-526a1f023efd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.210317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.211557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.212218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.212771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.215647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:58.227248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.229094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:25:58.249502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:58.250461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:58.251348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:25:58.251348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:25:58.251348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:25:58.251348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:58.253956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 342/705] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:58.255308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.258132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b005e1-b973-4f2d-9479-d7cb3101c462 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.295042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.295856 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.296136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.296410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.297492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.298446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:58.312501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:58.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:25:58.314431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:58.318697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 364/706] 10.222.0.25 () {56 vars in 1129 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:58.319386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.364063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:58.377796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c82e0-cae5-4eb5-bdb5-92a3cadb8125 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.415719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.416811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.417895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.418375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.419488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:58.419992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.421664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.425242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642ef7-3740-4d79-a25b-b02f73122f40 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.434895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 343/707] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:58.435987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.465549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.466260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.466544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.466824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.467637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:58.467918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.469412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.482776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 365/708] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:58.483671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.490113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:58.551178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc04f10-8530-4448-8eb6-717bfab64e06 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.587794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.588626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.589170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.589526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.590458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:58.590868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.592340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.594047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:58.597159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963b17e-7801-4827-a731-044692fc0987 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.599402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 344/709] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:58.600646 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.646808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.647604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.647990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.648363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.656507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.658185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:25:58.673342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:58.674486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:25:58.675621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:25:58.677253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 366/710] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:58.678161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.719922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f355c-a51a-4378-a4f5-e37344f921b1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.732881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:58.760536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.762236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.762656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.763072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.763913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.765678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.785169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7777c-1b3f-40a1-b548-d8130c40fe01 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.788491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 345/711] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:25:58 2022] POST /identity/v3/users => generated 254 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:25:58.789760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:58.824408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.825444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.825679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.826093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.826955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:58.827342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.828964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.835666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 367/712] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:58.918075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa173f1-e82e-4ffb-9ec6-47a1fafbb622 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:58.959422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a6faf-45a4-460d-8072-8ac927acf7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:58.960467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a6faf-45a4-460d-8072-8ac927acf7be None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:58.960908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a6faf-45a4-460d-8072-8ac927acf7be None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:58.961332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a6faf-45a4-460d-8072-8ac927acf7be None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:58.962384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a6faf-45a4-460d-8072-8ac927acf7be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:58.962848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a6faf-45a4-460d-8072-8ac927acf7be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:58.965055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a6faf-45a4-460d-8072-8ac927acf7be None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:58.980904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 346/713] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:58.986482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:59.020190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:59.114775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07e6c-b560-43bf-9a1a-0d1c158b088b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:59.161028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eca2c-df04-4c7d-a18a-4fc72993ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:59.161994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eca2c-df04-4c7d-a18a-4fc72993ecdb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:59.162283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eca2c-df04-4c7d-a18a-4fc72993ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:59.162560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eca2c-df04-4c7d-a18a-4fc72993ecdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:59.163416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eca2c-df04-4c7d-a18a-4fc72993ecdb None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=804a35678c91438eae36f61c1de8c579) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:59.163733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eca2c-df04-4c7d-a18a-4fc72993ecdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:59.165182 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eca2c-df04-4c7d-a18a-4fc72993ecdb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:59.172457 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 368/714] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:25:58 2022] GET /identity/v3/role_assignments?user.id=804a35678c91438eae36f61c1de8c579&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:59.205757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7990-7c7a-45aa-85fe-688b380cfdf4 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:59.206426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7990-7c7a-45aa-85fe-688b380cfdf4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:59.207018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7990-7c7a-45aa-85fe-688b380cfdf4 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:59.209077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 347/715] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:25:59.214997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8199-3414-4004-b993-9eb7aff5dfac None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:59.215270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8199-3414-4004-b993-9eb7aff5dfac None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:59.215487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8199-3414-4004-b993-9eb7aff5dfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:59.235752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-88ae8199-3414-4004-b993-9eb7aff5dfac None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:25:59.237000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-88ae8199-3414-4004-b993-9eb7aff5dfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:25:59.332666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 369/716] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:25:59.375868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7990-7c7a-45aa-85fe-688b380cfdf4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:59.416123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:59.517892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7990-7c7a-45aa-85fe-688b380cfdf4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:59.568379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:59.569190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:59.569475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:59.569754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] PATH_INFO: `/v3/services/network` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:59.570441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:59.571809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:25:59.592335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:25:59.594386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:25:59.596061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 348/717] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:25:59 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:25:59.603725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8199-3414-4004-b993-9eb7aff5dfac None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:25:59.671350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:25:59.789866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8199-3414-4004-b993-9eb7aff5dfac None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:25:59.868463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06ef09-90d2-420a-8612-4c509c88a6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:25:59.869147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06ef09-90d2-420a-8612-4c509c88a6cf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:25:59.869434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06ef09-90d2-420a-8612-4c509c88a6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:25:59.869712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06ef09-90d2-420a-8612-4c509c88a6cf None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:25:59.871007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06ef09-90d2-420a-8612-4c509c88a6cf None admin] RBAC: Adding query filter params (name=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:25:59.871318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06ef09-90d2-420a-8612-4c509c88a6cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:25:59.872695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06ef09-90d2-420a-8612-4c509c88a6cf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:25:59.882059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 370/718] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:25:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:25:59.887755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:00.012479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:00.023655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc16f8-fc0f-4ddf-8e2d-3d1742456a61 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:00.071252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b55840-beff-40d9-9fe2-906cd8ff4390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:00.071510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b55840-beff-40d9-9fe2-906cd8ff4390 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:00.071647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b55840-beff-40d9-9fe2-906cd8ff4390 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:00.071926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b55840-beff-40d9-9fe2-906cd8ff4390 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:00.072648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b55840-beff-40d9-9fe2-906cd8ff4390 None admin] RBAC: Adding query filter params (type=network) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:00.072960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b55840-beff-40d9-9fe2-906cd8ff4390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:00.074422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b55840-beff-40d9-9fe2-906cd8ff4390 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:00.082745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 349/719] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:25:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:00.469910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba62f7a-8014-4f2e-ad0c-46a46f420e94 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:00.470388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba62f7a-8014-4f2e-ad0c-46a46f420e94 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:00.470388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba62f7a-8014-4f2e-ad0c-46a46f420e94 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:00.471784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 371/720] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:00.477470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5797e9-b28a-4a4f-b2b1-1a1a8c38a516 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:00.477771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5797e9-b28a-4a4f-b2b1-1a1a8c38a516 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:00.478058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5797e9-b28a-4a4f-b2b1-1a1a8c38a516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:00.497007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3e5797e9-b28a-4a4f-b2b1-1a1a8c38a516 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:00.498235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3e5797e9-b28a-4a4f-b2b1-1a1a8c38a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:00.582552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 350/721] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:26:00.619822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba62f7a-8014-4f2e-ad0c-46a46f420e94 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:00.728884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:00.758834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba62f7a-8014-4f2e-ad0c-46a46f420e94 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:00.805261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:00.806093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:00.806381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:00.806660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:00.817790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:00.825407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:00.826332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:00.827171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 372/722] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:00.834402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5797e9-b28a-4a4f-b2b1-1a1a8c38a516 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:00.951353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:00.974707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5797e9-b28a-4a4f-b2b1-1a1a8c38a516 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:01.030264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:01.030955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.031242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.031514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.032276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:01.032581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:01.033996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:01.043993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 351/723] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:01.049354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:01.207731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624db3ac-c33a-4804-8285-257b77985aa1 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:01.249022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:01.249832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.250137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.250412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.264985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:01.266474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:01.271397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:01.296158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:01.297103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:01.298145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:01.298145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:01.298145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:01.298145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:01.300609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 373/724] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:01.306446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:01.442460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae46da-7915-4859-a768-8fb0acd34523 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:01.484298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0447d-1007-4892-a89a-be8ea2dc2b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:01.485069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0447d-1007-4892-a89a-be8ea2dc2b18 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.485359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0447d-1007-4892-a89a-be8ea2dc2b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.485637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0447d-1007-4892-a89a-be8ea2dc2b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.486399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0447d-1007-4892-a89a-be8ea2dc2b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:01.486709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0447d-1007-4892-a89a-be8ea2dc2b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:01.488080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0447d-1007-4892-a89a-be8ea2dc2b18 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:01.505926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 352/725] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:01.512183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:01.605666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814d8af-b29f-46fd-8443-5d12588e84ac None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.605983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814d8af-b29f-46fd-8443-5d12588e84ac None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.606176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814d8af-b29f-46fd-8443-5d12588e84ac None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.613950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 353/726] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:01.617849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8891d4c-c767-463e-9d9d-0a852d9f0fd2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.618139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8891d4c-c767-463e-9d9d-0a852d9f0fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.618419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8891d4c-c767-463e-9d9d-0a852d9f0fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.639538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:01.650786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c8891d4c-c767-463e-9d9d-0a852d9f0fd2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:01.652535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c8891d4c-c767-463e-9d9d-0a852d9f0fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:01.703004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc78d0-9b1d-41ac-be7c-82e6d6b53f3f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:01.749409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 354/727] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:26:01.772254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:01.773040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8891d4c-c767-463e-9d9d-0a852d9f0fd2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:01.773154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.773358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.773635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.782984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:01.785581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:01.815999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:01.817024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:01.818595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 374/728] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:01.822647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:01.886841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:01.913170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8891d4c-c767-463e-9d9d-0a852d9f0fd2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:01.934967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598af5e5-c436-4637-a645-4c31272b8a6e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:01.952412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b79af-034a-4178-bffb-668740078676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:01.953222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b79af-034a-4178-bffb-668740078676 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.953511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b79af-034a-4178-bffb-668740078676 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.953791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b79af-034a-4178-bffb-668740078676 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.956414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:01.965454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b79af-034a-4178-bffb-668740078676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:01.972703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63bb32-101c-43ea-8a25-005d57e05099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-847b79af-034a-4178-bffb-668740078676 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:01.973197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:01.974187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:26:01.975324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:01.976154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bb32-101c-43ea-8a25-005d57e05099 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.976154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bb32-101c-43ea-8a25-005d57e05099 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.976154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63bb32-101c-43ea-8a25-005d57e05099 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.976374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 355/729] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:26:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:01.976374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea2b8-95fc-4503-93e9-64a47812d2a2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.976374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea2b8-95fc-4503-93e9-64a47812d2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.976561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea2b8-95fc-4503-93e9-64a47812d2a2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.977900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 356/730] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:01.979700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63bb32-101c-43ea-8a25-005d57e05099 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:01.980024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63bb32-101c-43ea-8a25-005d57e05099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:01.981439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63bb32-101c-43ea-8a25-005d57e05099 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:01.982731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530b990-9c9d-4b26-8484-dd8f9d161c9d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:01.983146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530b990-9c9d-4b26-8484-dd8f9d161c9d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:01.983317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530b990-9c9d-4b26-8484-dd8f9d161c9d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:01.985171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 357/731] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:01.985291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530b990-9c9d-4b26-8484-dd8f9d161c9d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.003705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 375/732] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Jun 17 11:26:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:02.004756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5466-99bf-478a-b7ec-51f0199498cb None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.005042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5466-99bf-478a-b7ec-51f0199498cb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.005314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b5466-99bf-478a-b7ec-51f0199498cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.021227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a89b5466-99bf-478a-b7ec-51f0199498cb None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:02.022423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a89b5466-99bf-478a-b7ec-51f0199498cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:02.092422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 376/733] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:26:02.093489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42840778-38a2-4146-90b1-14c81ba49355 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.093761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42840778-38a2-4146-90b1-14c81ba49355 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.094060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42840778-38a2-4146-90b1-14c81ba49355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.107152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530b990-9c9d-4b26-8484-dd8f9d161c9d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.119104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-42840778-38a2-4146-90b1-14c81ba49355 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:02.120838 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-42840778-38a2-4146-90b1-14c81ba49355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:02.146833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.147458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.147743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.148017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.148771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:02.149077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:02.150496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:02.159055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:02.159436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 358/734] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:26:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:02.160231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.200388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 377/735] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:26:02.201589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42840778-38a2-4146-90b1-14c81ba49355 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.223937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:02.269864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97381f23-587a-412f-9852-23f240d6baa8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.305363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.306056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.306343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.306619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.316701 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42840778-38a2-4146-90b1-14c81ba49355 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.352677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.353477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.353757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.354058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.365591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:02.374329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:02.376215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 378/736] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:02.376912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.382703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:02.384173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:02.408643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:02.442792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:02.490840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 359/737] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 17 11:26:02.490840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.491111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e3baa-73e7-42bf-8218-1f04674fa799 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.526825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.527620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.527900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.528173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.539717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:02.541404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:02.559770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:02.560679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:02.561569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:02.561569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:02.561569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:02.561569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:02.561569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 379/738] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:02.562307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.600386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf102a9-f878-4301-9725-fa67c31adaeb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.644782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.645589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.645907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.646184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.646928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:02.648334 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:02.665285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 360/739] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:02 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:02.666086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.670673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:02.692142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ca818-41f5-41a3-9fc5-14cd6c1b3da6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.702009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:02.745551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.746324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.746628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.746886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.747672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:02.747986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:02.749474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:02.763480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 380/740] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:02.763480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.810866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19261141-7567-45ee-9c31-99cead7f6205 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.852030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3f49e-1abb-45b1-ac92-7fc9e9c1e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.852850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3f49e-1abb-45b1-ac92-7fc9e9c1e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.853263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3f49e-1abb-45b1-ac92-7fc9e9c1e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.853545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3f49e-1abb-45b1-ac92-7fc9e9c1e5a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.865519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3f49e-1abb-45b1-ac92-7fc9e9c1e5a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:02.867278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3f49e-1abb-45b1-ac92-7fc9e9c1e5a3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:02.879948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602416-0faf-4263-9d4f-e1e09d374bfb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-88c3f49e-1abb-45b1-ac92-7fc9e9c1e5a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:02.885077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:02.886179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:02.887085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:02.887085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:02.887085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:02.887085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:02.887085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 361/741] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:02.887907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd696-60e7-480a-ae0d-1d7fda6d931d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.888198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd696-60e7-480a-ae0d-1d7fda6d931d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.888478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd696-60e7-480a-ae0d-1d7fda6d931d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.893669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 362/742] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:02.894489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fd696-60e7-480a-ae0d-1d7fda6d931d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.925006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:02.925502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:02.925791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:02.926113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:02.926851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:02.927164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:02.928535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:02.944406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 381/743] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:02.944406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:02.945578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:02.974811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:03.011705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fd696-60e7-480a-ae0d-1d7fda6d931d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.046720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af375b8-3057-4204-933b-7ff0bddcbb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.047522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af375b8-3057-4204-933b-7ff0bddcbb18 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.047806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af375b8-3057-4204-933b-7ff0bddcbb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.048080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af375b8-3057-4204-933b-7ff0bddcbb18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.051776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3bb8-b8ee-4f79-a1a0-1b712d213041 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.059609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af375b8-3057-4204-933b-7ff0bddcbb18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.061079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af375b8-3057-4204-933b-7ff0bddcbb18 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5af375b8-3057-4204-933b-7ff0bddcbb18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:03.078908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:03.079786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:03.080675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:03.080675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:03.080675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:03.080675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:03.080675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 363/744] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:03.081374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd79d7-d22a-4e13-88e5-1410faf44b6b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.081651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd79d7-d22a-4e13-88e5-1410faf44b6b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.081850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd79d7-d22a-4e13-88e5-1410faf44b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.093524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.094230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.094517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.094794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.095523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:03.095837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.097198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.099551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cccd79d7-d22a-4e13-88e5-1410faf44b6b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:03.100760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cccd79d7-d22a-4e13-88e5-1410faf44b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:03.114939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 382/745] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:03.114939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.176533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 364/746] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 17 11:26:03.177610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd79d7-d22a-4e13-88e5-1410faf44b6b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.187785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:03.232060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9674976-862a-46bc-937f-2d19423e525a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.235534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:03.267304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.268273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.268669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.269053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.276839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.278480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.286005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd79d7-d22a-4e13-88e5-1410faf44b6b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:03.294438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:03.295671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:03.296841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:03.297381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 383/747] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:03.297721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.328377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.329171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.329458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.329861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.330709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:03.330904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.332283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.347858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 365/748] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:03.347858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.410573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a15c9a-a68f-44c3-ac16-d3d40d02b186 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.434733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:03.447244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.448047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.448386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.448666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.455659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.457113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.462728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bb8e0-38ed-4978-a92c-0df2a8e5ce21 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:03.471375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:03.472278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:03.473180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 384/749] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:03.473673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.481740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:03.505808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.506511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.506806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.507069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.507786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.509143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.533878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 366/750] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:03 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:03.533878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.620063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2d52-41f0-46d3-a58e-af09fb12785b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.673728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080836b4-dd46-46f1-89ac-8bf09b2c796b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.688247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.688991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.689390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.690026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.691607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:03.692001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.694169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.717544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.718534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.718817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.719095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.720879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 385/751] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:03.721878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.728238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.729726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:03.746061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:03.747370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 367/752] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:03.748481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.778105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:03.787418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:03.846263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695cc74-e93c-4888-9461-a39b0bb91d10 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.881485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.882516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.882984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.883448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:03.884451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:03.884945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:03.887296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:03.888866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf093b22-2f28-4d2a-991a-52dfd6a4d48b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:03.899932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 386/753] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:03.901065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:03.938126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:03.938854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:03.939077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:03.939353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.011106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:04.015317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886ea85-02f1-41e4-9b3e-0ecb7e6ceaec None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.030554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.032537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.053684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.054391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.054672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.054943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.055782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:04.056092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.058072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.070716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:04.082257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 387/754] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:04.083357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.154920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 368/755] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:04.154920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.197230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419cece-f59f-46cc-8ef1-59e7caf2cc17 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.234688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.235586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.236170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.236611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.237704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=fd3119ad342840ddb0258e124be55597) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:04.238192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.239888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.246827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 388/756] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/role_assignments?user.id=fd3119ad342840ddb0258e124be55597&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:04.247963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.256391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c660db-05e7-40b0-9bde-03a84fc1f982 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.276274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:04.288865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:04.298495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.299448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.299833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.300217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.313063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.315224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:04.335535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:04.336702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:04.337594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:04.337594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:04.337594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:04.337594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:04.337815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 369/757] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:04.345424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.372224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539d512-ced5-4e54-b726-f162c2563e0a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.430461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.431283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.431570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.431850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.451757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.453173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:04.496695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:04.497637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:04.498554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:04.498554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:04.498554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:04.498554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:04.499189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 389/758] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:04.506985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.528283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6545b6c-9076-4d75-b253-f39fc221977d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.575921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.576600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.576883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.577161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.579062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:04.579379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.580767 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.602301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 370/759] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:04.608398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.630931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:04.681297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb68f1-b987-48b7-8c07-fe0d77a33401 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.692188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:04.755133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.755813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.756096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.756371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.757105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:04.757414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.758810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.785479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 390/760] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:04.794407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.811691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6566a-3427-490c-a3c8-c5264048b370 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.875488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912e6ba-efb2-485c-a3fe-a15609465a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.876047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912e6ba-efb2-485c-a3fe-a15609465a1a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.876331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912e6ba-efb2-485c-a3fe-a15609465a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.876608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912e6ba-efb2-485c-a3fe-a15609465a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.885096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912e6ba-efb2-485c-a3fe-a15609465a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:04.887063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912e6ba-efb2-485c-a3fe-a15609465a1a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2912e6ba-efb2-485c-a3fe-a15609465a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:04.905755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:04.906704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:04.907582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 371/761] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:04.908735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9698e1-4164-4cb1-b60b-c84099dbb25a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.909021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9698e1-4164-4cb1-b60b-c84099dbb25a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.909459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9698e1-4164-4cb1-b60b-c84099dbb25a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.913964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 372/762] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:04.914904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9698e1-4164-4cb1-b60b-c84099dbb25a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:04.950134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd175-61d4-4fae-a8e4-1d2a7827ba5d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:04.993983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61523ff8-636f-4667-b937-959d5bbd1001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:04.995165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61523ff8-636f-4667-b937-959d5bbd1001 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:04.995645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61523ff8-636f-4667-b937-959d5bbd1001 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:04.996134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61523ff8-636f-4667-b937-959d5bbd1001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:04.999412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:05.006523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61523ff8-636f-4667-b937-959d5bbd1001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.008437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61523ff8-636f-4667-b937-959d5bbd1001 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-61523ff8-636f-4667-b937-959d5bbd1001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:05.025062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:05.026018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:05.027430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:05.034984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 391/763] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:05.036869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f66cd1-ecd0-43ec-82ad-a924d9048fc6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.037413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f66cd1-ecd0-43ec-82ad-a924d9048fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.037908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f66cd1-ecd0-43ec-82ad-a924d9048fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.045122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:05.051000 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9698e1-4164-4cb1-b60b-c84099dbb25a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:05.060232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-86f66cd1-ecd0-43ec-82ad-a924d9048fc6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:05.061758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-86f66cd1-ecd0-43ec-82ad-a924d9048fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:05.088261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:05.089470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.089753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.090082 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.090978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:05.091623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.092868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.106326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 373/764] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:05.107234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:05.136950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 392/765] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:05.138199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f66cd1-ecd0-43ec-82ad-a924d9048fc6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:05.240182 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f66cd1-ecd0-43ec-82ad-a924d9048fc6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:05.244465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a327f2-f615-4b27-b596-e89f78f6c081 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:05.270297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:05.278439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:05.279791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.280369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.280931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/804a35678c91438eae36f61c1de8c579/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.281393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:05.282252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.282539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.282815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.283607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:05.283917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.284515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=804a35678c91438eae36f61c1de8c579, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.285362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.286348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.300036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 374/766] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:05.305179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:05.314031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 393/767] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:26:05 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/804a35678c91438eae36f61c1de8c579/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:26:05.315136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:05.331942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:05.421259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ef44b-216e-46dc-a0bd-fe997439c19c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:05.459704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:05.460904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.461385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.461904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.462756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.465020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:05.478288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:05.479276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:26:05.480101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:05.482994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 375/768] 10.222.0.25 () {56 vars in 1129 bytes} [Fri Jun 17 11:26:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:05.489203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:05.508240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a442-7f74-4cae-aad5-b3600e052658 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:05.586829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:05.606188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48efe2-942e-4fc3-b52a-c76b83292195 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:05.631071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:05.631071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.631071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.631071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.631837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=84e3617bb236496783475fad6693d9af) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:05.633029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.634519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.647809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 394/769] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:05 2022] GET /identity/v3/role_assignments?user.id=84e3617bb236496783475fad6693d9af&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:05.665068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:05.666102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.666560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.667043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.668991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:05.669864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.672555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.679918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 376/770] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Jun 17 11:26:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:05.685822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:05.754537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:05.800156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad85d01-28a5-4604-969b-1315cf9d3fdb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:05.837934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:05.838541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:05.838854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:05.839130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:05.839850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:05.840155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:05.841545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:05.847536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 395/771] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Jun 17 11:26:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:05.859010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:05.974873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79aa35-5d4d-468a-9c58-8d7b55d6fd43 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:06.013993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dde03-843c-41e3-a6d6-c12d99453e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:06.014673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dde03-843c-41e3-a6d6-c12d99453e6d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.014959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dde03-843c-41e3-a6d6-c12d99453e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.015238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dde03-843c-41e3-a6d6-c12d99453e6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.016044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dde03-843c-41e3-a6d6-c12d99453e6d None admin] RBAC: Adding query filter params (interface=public, service_id=8397a4e660364118a0a3aeffdcd7e86f, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:06.016358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dde03-843c-41e3-a6d6-c12d99453e6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:06.017735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dde03-843c-41e3-a6d6-c12d99453e6d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:06.026017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 377/772] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:26:05 2022] GET /identity/v3/endpoints?service_id=8397a4e660364118a0a3aeffdcd7e86f&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:06.034977 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:06.035573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f16e5b-d269-48be-9875-13f45cd3e865 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.035868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f16e5b-d269-48be-9875-13f45cd3e865 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.036150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f16e5b-d269-48be-9875-13f45cd3e865 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.038316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 378/773] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:06.044342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469375f-e8b5-4eeb-b034-9749f4e8f894 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.044634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469375f-e8b5-4eeb-b034-9749f4e8f894 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.044915 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469375f-e8b5-4eeb-b034-9749f4e8f894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.064462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8469375f-e8b5-4eeb-b034-9749f4e8f894 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:06.065676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8469375f-e8b5-4eeb-b034-9749f4e8f894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:06.074738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:06.168360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 379/774] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:06.199628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98dff-acfd-41ff-b8e8-ca0651ab101d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:06.202029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469375f-e8b5-4eeb-b034-9749f4e8f894 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:06.218372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:06.240017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:06.240756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.241043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.241322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.242096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:06.243457 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:06.250484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 396/775] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:06 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:06.385226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469375f-e8b5-4eeb-b034-9749f4e8f894 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:06.439076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:06.439820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.440105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.440383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] PATH_INFO: `/v3/services/network` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.441047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:06.446676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:06.459577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:06.460655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:06.461501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 380/776] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:06 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:06.471068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:06.593302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575b63a-e375-4265-9337-004919cc12ba None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:06.637372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5a8b9-b5bc-4302-b084-316c34b4039d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:06.637372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5a8b9-b5bc-4302-b084-316c34b4039d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.637975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5a8b9-b5bc-4302-b084-316c34b4039d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.637975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5a8b9-b5bc-4302-b084-316c34b4039d None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.641869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5a8b9-b5bc-4302-b084-316c34b4039d None admin] RBAC: Adding query filter params (name=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:06.641869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5a8b9-b5bc-4302-b084-316c34b4039d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:06.641869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b5a8b9-b5bc-4302-b084-316c34b4039d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:06.645702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 397/777] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:06.650914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:06.658228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dd235-d604-4db5-8957-eede88454bf0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.658569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dd235-d604-4db5-8957-eede88454bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.659824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dd235-d604-4db5-8957-eede88454bf0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.661201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 398/778] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:06.669250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:06.671534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21412454-fae6-4183-ab9e-6f886542d61d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.671872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21412454-fae6-4183-ab9e-6f886542d61d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.672161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21412454-fae6-4183-ab9e-6f886542d61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.705220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-21412454-fae6-4183-ab9e-6f886542d61d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:06.707659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-21412454-fae6-4183-ab9e-6f886542d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:06.775847 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:06.840983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 399/779] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:06.848221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0441d-4911-4804-add2-f9f8ae2ca78b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:06.895765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:06.896845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:06.897338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:06.897887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:06.898936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] RBAC: Adding query filter params (type=network) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:06.899472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:06.901719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:06.908325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 381/780] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:06.910042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21412454-fae6-4183-ab9e-6f886542d61d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:06.920831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:07.070185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:07.086546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21412454-fae6-4183-ab9e-6f886542d61d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:07.124036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:07.125216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:07.125735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:07.126256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:07.139035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:07.147268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:07.148302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:07.149132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:07.149808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666759d-8b29-4436-a23f-7997825868cb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:07.152445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 400/781] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:07.158377 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:07.194203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:07.207204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:07.207887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:07.208167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:07.208444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:07.209268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] RBAC: Adding query filter params (interface=public, service_id=46540103cc8143719bd453104d3b6d96, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:07.209540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:07.210938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:07.216493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 382/782] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:26:06 2022] GET /identity/v3/endpoints?service_id=46540103cc8143719bd453104d3b6d96&interface=public®ion_id=RegionOne => generated 194 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:07.221656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:07.274208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129904b-7747-4b52-9b29-f768661a0b2f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:07.312172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:07.313247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:07.313736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:07.314260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:07.315383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:07.315916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:07.317660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:07.324802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79014888-f7b7-4545-b919-455c626a17c5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:07.328362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 401/783] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:07.334899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:07.360980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:07.361660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:07.361971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:07.362283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:07.362969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:07.364347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:07.365423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:07.371632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 383/784] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:07 2022] GET /identity/v3/services => generated 772 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:07.510238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:07.522549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ac47-d6c3-4bd7-8181-28daf30ce1df None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:07.561613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:07.562471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:07.562899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:07.563238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:07.574967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:07.576365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:07.594885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:07.595850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:07.596760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:07.596760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:07.596760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:07.596760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:07.596760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 402/785] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:07.603185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:07.723555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a044e-133d-4d7b-bad7-c768a1f90ce1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:07.759274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:07.759949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:07.760230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:07.761043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:07.761822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:07.762155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:07.763525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:07.778255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 384/786] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:07.787226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:07.945137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a2174-53af-4f39-8f43-4cba114c9fac None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:07.993371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7e154-fa3e-4191-8d0c-58bc2ae61cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:07.994209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7e154-fa3e-4191-8d0c-58bc2ae61cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:07.994496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7e154-fa3e-4191-8d0c-58bc2ae61cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:07.995298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7e154-fa3e-4191-8d0c-58bc2ae61cf0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.003154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7e154-fa3e-4191-8d0c-58bc2ae61cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:08.004664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7e154-fa3e-4191-8d0c-58bc2ae61cf0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-28c7e154-fa3e-4191-8d0c-58bc2ae61cf0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:08.023045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:08.024027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:08.024972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 403/787] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:08.034950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:08.131533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:08.201024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77629d7c-c373-4643-858e-c799ad075d3b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:08.213684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3dc13-1750-4028-aeb1-49d7afedc0ca None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.214018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3dc13-1750-4028-aeb1-49d7afedc0ca None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.214298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3dc13-1750-4028-aeb1-49d7afedc0ca None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.215929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 404/788] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:08.220812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1f425-456a-481f-843b-85f44b3a5aed None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.221103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1f425-456a-481f-843b-85f44b3a5aed None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.221386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1f425-456a-481f-843b-85f44b3a5aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.233670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:08.247807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-61e1f425-456a-481f-843b-85f44b3a5aed None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:08.249019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-61e1f425-456a-481f-843b-85f44b3a5aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:08.252957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:08.253644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.253966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.254249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.255099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:08.255413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:08.256863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:08.286898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 385/789] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Jun 17 11:26:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:08.297740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:08.383748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 405/790] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:08.406988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1f425-456a-481f-843b-85f44b3a5aed None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:08.508525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74419887-ff48-4532-a33a-b4fb5ffaa809 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:08.517800 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1f425-456a-481f-843b-85f44b3a5aed None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:08.547714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:08.572330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:08.573196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.573483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.573761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.584718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:08.585557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.585871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.586141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.586243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:08.600653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:08.609068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:08.610077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:08.610926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:08.613394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 406/791] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:08.618717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:08.620242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:08.622866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:08.651847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:08.654048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:08.656254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:08.656254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:08.656254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:08.656254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:08.660916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 386/792] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:08.662140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:08.746244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d33f31-467c-4999-b21a-3272e8ed17b8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:08.783308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e4344-1dd8-4548-8e89-bac0aa4f655a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:08.783908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e4344-1dd8-4548-8e89-bac0aa4f655a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.784199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e4344-1dd8-4548-8e89-bac0aa4f655a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.784480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e4344-1dd8-4548-8e89-bac0aa4f655a None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.785243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e4344-1dd8-4548-8e89-bac0aa4f655a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:08.785557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e4344-1dd8-4548-8e89-bac0aa4f655a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:08.787262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e4344-1dd8-4548-8e89-bac0aa4f655a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:08.793603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c447aa-a52a-4628-877c-265b5e3dde58 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:08.801542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 407/793] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:08.803186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad228f81-2cb3-4097-a719-d704ba0d52d0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.803478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad228f81-2cb3-4097-a719-d704ba0d52d0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.803761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad228f81-2cb3-4097-a719-d704ba0d52d0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.805380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 408/794] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:08.806048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad228f81-2cb3-4097-a719-d704ba0d52d0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:08.848012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed523c-0753-4520-9346-3a36f8395a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:08.848881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed523c-0753-4520-9346-3a36f8395a53 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.849261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed523c-0753-4520-9346-3a36f8395a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.849547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed523c-0753-4520-9346-3a36f8395a53 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.850309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed523c-0753-4520-9346-3a36f8395a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:08.850789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed523c-0753-4520-9346-3a36f8395a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:08.853086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed523c-0753-4520-9346-3a36f8395a53 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:08.869992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 387/795] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:08.871273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911f4b-674b-4c86-a81e-99609a437438 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.871564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911f4b-674b-4c86-a81e-99609a437438 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.871847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911f4b-674b-4c86-a81e-99609a437438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:08.879263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:08.892172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a4911f4b-674b-4c86-a81e-99609a437438 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:08.893386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4911f4b-674b-4c86-a81e-99609a437438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:08.917420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:08.930661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad228f81-2cb3-4097-a719-d704ba0d52d0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:08.965216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 388/796] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:08.966417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4911f4b-674b-4c86-a81e-99609a437438 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:08.984862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655af4b-3368-48d1-a563-6dbbba555ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:08.986059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655af4b-3368-48d1-a563-6dbbba555ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:08.986549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655af4b-3368-48d1-a563-6dbbba555ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:08.987026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655af4b-3368-48d1-a563-6dbbba555ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.001287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655af4b-3368-48d1-a563-6dbbba555ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.003103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655af4b-3368-48d1-a563-6dbbba555ea9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b655af4b-3368-48d1-a563-6dbbba555ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:09.021663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:09.022640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:09.023554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:09.023554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:09.023554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:09.023554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:09.026557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 409/797] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:09.028014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e450-4258-4b9e-be73-c7be8f6ff0e3 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.028379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e450-4258-4b9e-be73-c7be8f6ff0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.028730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8e450-4258-4b9e-be73-c7be8f6ff0e3 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.033934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 410/798] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:09.033934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8e450-4258-4b9e-be73-c7be8f6ff0e3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.093399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4911f4b-674b-4c86-a81e-99609a437438 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.127286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.128101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.128391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.128672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.136075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.137527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.140243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8e450-4258-4b9e-be73-c7be8f6ff0e3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.149273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:09.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:09.152843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:09.153731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 389/799] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:09.154585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.176588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:09.184731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b251c70-50e6-449b-96f6-ece12d9f5425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.185557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b251c70-50e6-449b-96f6-ece12d9f5425 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.185833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b251c70-50e6-449b-96f6-ece12d9f5425 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.186128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b251c70-50e6-449b-96f6-ece12d9f5425 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.196666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b251c70-50e6-449b-96f6-ece12d9f5425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0b251c70-50e6-449b-96f6-ece12d9f5425 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:09.204936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:09.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:09.206840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 411/800] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:09.207524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f8fc-f94f-4759-8db1-bfd76baa3599 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.207777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f8fc-f94f-4759-8db1-bfd76baa3599 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.208061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f8fc-f94f-4759-8db1-bfd76baa3599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.226308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-96a1f8fc-f94f-4759-8db1-bfd76baa3599 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:09.227541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-96a1f8fc-f94f-4759-8db1-bfd76baa3599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:09.270814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c410c-a50f-4150-b924-412e4e733869 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.301967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 412/801] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:09.302666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f8fc-f94f-4759-8db1-bfd76baa3599 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.316222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.317310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.317801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.318563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.320028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:09.320786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.323645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.339821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 390/802] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:09.340972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.406524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:09.411172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f8fc-f94f-4759-8db1-bfd76baa3599 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.425018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:09.450517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.451198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.451446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4e44-5913-4019-9e61-9de66f923d2c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.451589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.451761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.452538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:09.452859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.454319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.467019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 413/803] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:09.467919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.493600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.494413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.495346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.495631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.496382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:09.496694 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.498084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.507844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 391/804] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:09.512007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.578057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69f00-aaba-41b0-b137-86ac7767f91c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.616664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836724f7-83a4-4577-b679-c599f035831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.617289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae2b61-2f5f-4165-9106-44d4d8e529bb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.617492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836724f7-83a4-4577-b679-c599f035831c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.617779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836724f7-83a4-4577-b679-c599f035831c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.618102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836724f7-83a4-4577-b679-c599f035831c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.618906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836724f7-83a4-4577-b679-c599f035831c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.623003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836724f7-83a4-4577-b679-c599f035831c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-836724f7-83a4-4577-b679-c599f035831c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:09.640842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:09.642863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:09.643724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 414/805] 10.222.0.25 () {56 vars in 1129 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:09.645650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836724f7-83a4-4577-b679-c599f035831c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.664533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.665343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.665638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.665946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.672765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:09.675604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.677056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.680431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:09.694452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:09.695450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:09.696426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 392/806] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:09.697001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.768479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836724f7-83a4-4577-b679-c599f035831c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.805357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.806289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.806570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.806844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/fd3119ad342840ddb0258e124be55597/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.809338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=fd3119ad342840ddb0258e124be55597, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.810852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.823400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 415/807] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:26:09 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/fd3119ad342840ddb0258e124be55597/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:26:09.824187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.829286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c175f6-9f8a-46af-88c5-c717fde470a1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.871279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:09.872103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:09.872389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:09.872842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:09.885825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:09.887358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:09.906199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:09.907196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:09.908111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:09.908111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:09.908111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:09.908111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:09.908111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 393/808] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:09.908396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:09.936879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:09.959896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eb0fd-e7c7-4bd1-b0e9-83b6d2947b18 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:09.972066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:10.000243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029da947-9f4c-406a-8d43-c9c2e7387ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.000929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029da947-9f4c-406a-8d43-c9c2e7387ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.001215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029da947-9f4c-406a-8d43-c9c2e7387ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.001496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029da947-9f4c-406a-8d43-c9c2e7387ce7 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.002319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029da947-9f4c-406a-8d43-c9c2e7387ce7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:10.002638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029da947-9f4c-406a-8d43-c9c2e7387ce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.004014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029da947-9f4c-406a-8d43-c9c2e7387ce7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.008740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 416/809] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:10.010041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5127b-f9b0-4225-b260-210306714c40 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.010605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5127b-f9b0-4225-b260-210306714c40 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.010730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5127b-f9b0-4225-b260-210306714c40 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.012133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 417/810] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:10.012855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5127b-f9b0-4225-b260-210306714c40 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.035181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b79d78-5cb2-4201-98c5-0cc769b2ad71 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.072857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.073539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.073826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.074135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.074985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:10.075300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.076745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.100597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 394/811] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:26:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:10.100597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.120868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5127b-f9b0-4225-b260-210306714c40 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.156909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97971fc-d107-4e09-bfdd-702b15a3073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.157689 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97971fc-d107-4e09-bfdd-702b15a3073f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.158273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97971fc-d107-4e09-bfdd-702b15a3073f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.158394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97971fc-d107-4e09-bfdd-702b15a3073f None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.159216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97971fc-d107-4e09-bfdd-702b15a3073f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:10.159466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97971fc-d107-4e09-bfdd-702b15a3073f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.160827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97971fc-d107-4e09-bfdd-702b15a3073f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.173971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 418/812] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:10.174924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54126345-ca9c-404e-ac70-a8f28bbb0bf9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.175216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54126345-ca9c-404e-ac70-a8f28bbb0bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.175495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54126345-ca9c-404e-ac70-a8f28bbb0bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.186158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:10.192992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-54126345-ca9c-404e-ac70-a8f28bbb0bf9 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:10.194245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-54126345-ca9c-404e-ac70-a8f28bbb0bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:10.216377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba410b2-c986-4942-a68b-68844a589cf3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.228154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:10.253465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.254169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.254450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.254779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.255960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:10.256272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.258157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.268942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 395/813] 10.222.0.25 () {56 vars in 1124 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:10.268942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.270871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 419/814] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:10.271620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54126345-ca9c-404e-ac70-a8f28bbb0bf9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.381679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f58209-b0fa-483d-b3eb-2b351a891ca4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.386363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54126345-ca9c-404e-ac70-a8f28bbb0bf9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.417911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.418578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.418930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.419219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.424196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.425011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.425297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.425574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.428840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:10.434067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.435769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:10.456010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:10.456960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 420/815] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:10.458543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.467547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:10.496750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.498241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.562461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d8864-2aa7-49ae-944f-d002a31ad735 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.599646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.600322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.600608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.600885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.605800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 396/816] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:10.605951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.606031 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.606103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.635150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 421/817] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:10.635150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.704924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:10.719760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:10.742727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5abe2-f1ea-4318-964e-bd6eca65f9af None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.788882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcfe68-faaf-4911-af0b-8e14d26d6f84 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.790436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.791492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.791877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.792228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.793647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.794644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:10.808561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:10.809593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 17 11:26:10.810450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:10.811187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 397/818] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:10.812030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.841030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.841608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.841929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.842208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.843237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:10.843533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:10.844951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:10.868876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 422/819] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:10.868876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:10.952483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0daca4-0382-4350-a7ee-901578b1ffb2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.987900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b2b20-55cb-4f48-bb2c-ff0777c9d0fb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:10.993741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:10.995408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:10.995921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:10.996398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:10.997272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:11.000903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:11.014411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.016336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.033020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.033703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.034031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.034312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.035026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] RBAC: Adding query filter params (name=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:11.035342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.036975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:11.037686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:11.038595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:11.039505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:11.039505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:11.039505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:11.039505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:11.041866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 398/820] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:11.042982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.045974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 423/821] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:11.052876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.160666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad4e843-14f3-4f70-ba4b-49c7d672e9b8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.190552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476864-584b-4f2b-8d6d-213075db0a86 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.199589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.200801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.201293 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.201770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.214123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.215934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.233275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.234032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.234321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.234598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.235306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] RBAC: Adding query filter params (type=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:11.235615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.236963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:11.237599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:11.238890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:11.239776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:11.239776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:11.239776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:11.239776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:11.247101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 399/823] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:11.247253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 424/823] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:11.248051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.248672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.284360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:11.298968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:11.368196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26aec5d-02c4-4a9d-b728-03245ec65f8c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.374747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0ef40-2afa-415d-8233-27f232f3163c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.410503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.411826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.412010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.412505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.412613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.412906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.413183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.413500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.413932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:11.414247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.415031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:11.415572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.415684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.417817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.437910 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 400/824] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:11.437910 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.438736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 425/825] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:11.441153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.541120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:11.547100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfd269-2bfd-42f1-96ca-d0897a0ae68e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.569661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:11.578598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8584a6f-67a0-4610-9db3-25b62d82f275 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.587958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.589425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.590030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.590511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.591631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.593200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.617412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.618301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.618575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.618872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.620693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 401/826] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:11.624659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.647976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.660139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:11.700963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:11.701970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:11.702936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 426/827] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:11.718978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.851464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cbc3a3-78d8-4da8-b22a-fb6dc07f0ca9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.894907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.895725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.896011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.896323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.904247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.905527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd04cd-5433-4cc7-af07-0ce880b04f91 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:11.906017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:11.930248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:11.931188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:11.932127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 402/828] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:11.938644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:11.948626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:11.949302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:11.949585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:11.949880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:11.953571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:11.953946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:11.955419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:11.958721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:11.975231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 427/829] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:11.983075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:12.006613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:12.085116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ccf52b-a9be-4fe2-95e8-aff492b4af1f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:12.109192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb23fc3-0cd7-4451-9207-2e79652303b1 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:12.122225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9806a60-2665-421b-9bd0-6cb5facbcd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:12.122893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9806a60-2665-421b-9bd0-6cb5facbcd5b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.123176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9806a60-2665-421b-9bd0-6cb5facbcd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.123451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9806a60-2665-421b-9bd0-6cb5facbcd5b None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.124248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9806a60-2665-421b-9bd0-6cb5facbcd5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:12.124558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9806a60-2665-421b-9bd0-6cb5facbcd5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:12.126185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9806a60-2665-421b-9bd0-6cb5facbcd5b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:12.138111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 403/830] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:12.139173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56ca6-dd12-4d0d-b017-89d53642d12b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.139460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56ca6-dd12-4d0d-b017-89d53642d12b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.139737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56ca6-dd12-4d0d-b017-89d53642d12b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.141087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 404/831] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:12.145619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56ca6-dd12-4d0d-b017-89d53642d12b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:12.148648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9332f-4439-40c7-a875-1d959d2f094c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:12.150345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9332f-4439-40c7-a875-1d959d2f094c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.150900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9332f-4439-40c7-a875-1d959d2f094c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.151871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9332f-4439-40c7-a875-1d959d2f094c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.152900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9332f-4439-40c7-a875-1d959d2f094c None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=804a35678c91438eae36f61c1de8c579) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:12.153509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9332f-4439-40c7-a875-1d959d2f094c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:12.155584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9332f-4439-40c7-a875-1d959d2f094c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:12.162505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 428/832] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:11 2022] GET /identity/v3/role_assignments?user.id=804a35678c91438eae36f61c1de8c579&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:12.167299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029bdf0-c53e-486f-8d77-6934e9c87233 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.167591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029bdf0-c53e-486f-8d77-6934e9c87233 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.167873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029bdf0-c53e-486f-8d77-6934e9c87233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.192350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2029bdf0-c53e-486f-8d77-6934e9c87233 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:12.194014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2029bdf0-c53e-486f-8d77-6934e9c87233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:12.231005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:12.277433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56ca6-dd12-4d0d-b017-89d53642d12b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:12.294168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 429/833] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:12.318097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029bdf0-c53e-486f-8d77-6934e9c87233 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:12.325415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:12.326354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.326639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.328160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/84e3617bb236496783475fad6693d9af/roles/b3c5094c32f446d88c32145d8b8ae3cd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.333429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:12.348219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=84e3617bb236496783475fad6693d9af, role_id=b3c5094c32f446d88c32145d8b8ae3cd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:12.349569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:12.370296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 405/834] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:26:12 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/84e3617bb236496783475fad6693d9af/roles/b3c5094c32f446d88c32145d8b8ae3cd => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:26:12.457810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029bdf0-c53e-486f-8d77-6934e9c87233 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:12.496593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:12.497464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.497750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.498046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.511112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:12.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:12.520043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:12.520864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 430/835] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:12.528654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:12.645686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a3774-596f-4793-a043-c20984ca2116 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:12.672119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:12.689576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:12.690292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.691425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.691726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.692492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:12.692806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:12.694210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:12.703743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 406/836] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:12.709326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:12.786531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:12.818236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4274c-543c-48b8-9293-d8049c976e80 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:12.856437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d3d74-9990-4588-9ef8-c6b984cca6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:12.857646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3d74-9990-4588-9ef8-c6b984cca6aa None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.858168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3d74-9990-4588-9ef8-c6b984cca6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.858646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d3d74-9990-4588-9ef8-c6b984cca6aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.872197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d3d74-9990-4588-9ef8-c6b984cca6aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:12.874075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d3d74-9990-4588-9ef8-c6b984cca6aa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-fb6d3d74-9990-4588-9ef8-c6b984cca6aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:12.892668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:12.893564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:12.894498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:12.894498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:12.894498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:12.894498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:12.896712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 431/837] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:12.902710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:12.916153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c290bf-9a40-401c-9775-b230f84c2a89 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.917180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c290bf-9a40-401c-9775-b230f84c2a89 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.917258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c290bf-9a40-401c-9775-b230f84c2a89 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.922497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 432/838] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:12.924021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6c61-fc59-4b82-a30a-8c49dc13e006 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:12.924309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6c61-fc59-4b82-a30a-8c49dc13e006 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:12.924586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6c61-fc59-4b82-a30a-8c49dc13e006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:12.942763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-78cb6c61-fc59-4b82-a30a-8c49dc13e006 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:12.943938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-78cb6c61-fc59-4b82-a30a-8c49dc13e006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:13.012193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 433/839] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:13.030633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55150ae8-cc8f-43f5-94f9-8f48f8021e52 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:13.034482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb6c61-fc59-4b82-a30a-8c49dc13e006 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:13.072306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:13.073403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:13.074005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:13.074534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:13.075479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:13.075993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:13.077686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:13.087149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:13.093555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 407/840] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:13.099382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:13.140818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb6c61-fc59-4b82-a30a-8c49dc13e006 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:13.173144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:13.184099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:13.185382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:13.185821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:13.186274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:13.187079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:13.189473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:13.205959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:13.207164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:13.208200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:13.211603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 434/841] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:13.219775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:13.233398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ea2cd-599d-4cec-89a9-9f6466213a16 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:13.270491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:13.271451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:13.271804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:13.272143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:13.281635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:13.283232 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:13.303147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:13.304122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:13.305073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 408/842] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:13.310697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:13.353414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ea0e1-74ab-4beb-86c3-6d5a05aa8353 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:13.369341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:13.392997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:13.394091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:13.394442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:13.394932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:13.395815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:13.396211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:13.398170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:13.405878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 435/843] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:13.410917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:13.443284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3af7d-4c92-4966-b3da-9089d031bb7d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:13.446160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:13.539334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df7874-190a-4c71-abea-ae5330edc6ac None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:13.583263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:13.584319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:13.585316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:13.585793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:13.587001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:13.587760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:13.590617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:13.597900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 436/844] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:13.607746 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:13.608995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:13.609523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:13.610025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:13.611110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:13.611630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:13.613504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:13.634828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 409/845] 10.222.0.25 () {56 vars in 1135 bytes} [Fri Jun 17 11:26:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:13.643043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:13.693980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:13.758641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fc098-c44a-49de-9d44-26f1373743a4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:13.796375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:13.797035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:13.797319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:13.797594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:13.874254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:13.875692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:13.920504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:13.982502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 437/846] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:13.990944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:14.100719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d49cfa-4d7d-49d0-a52d-176a44f25079 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:14.136497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:14.137298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.137577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.137868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.148884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:14.150445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:14.168760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:14.169676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:14.170598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:14.170598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:14.170598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:14.170598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:14.170598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 410/847] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:14.175674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:14.281242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f5ca7-7769-4be7-806f-2ed284064c15 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:14.317961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d24793-4df7-40f5-892e-0319896c9fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:14.318980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d24793-4df7-40f5-892e-0319896c9fdf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.319489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d24793-4df7-40f5-892e-0319896c9fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.319969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d24793-4df7-40f5-892e-0319896c9fdf None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.320971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d24793-4df7-40f5-892e-0319896c9fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:14.321495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d24793-4df7-40f5-892e-0319896c9fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:14.323121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d24793-4df7-40f5-892e-0319896c9fdf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:14.335376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 438/848] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:14.341110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:14.358390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:14.439442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623802-357c-45ce-921a-756b12ea887e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.439731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623802-357c-45ce-921a-756b12ea887e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.440011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623802-357c-45ce-921a-756b12ea887e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.441379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 439/849] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:14.446467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600888cc-ea50-482d-9285-ebaf9cd84c19 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.446870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600888cc-ea50-482d-9285-ebaf9cd84c19 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.447251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600888cc-ea50-482d-9285-ebaf9cd84c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.452588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:14.460033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97ef96-7ea9-4361-b5e8-e3262e87ce20 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:14.465550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-600888cc-ea50-482d-9285-ebaf9cd84c19 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:14.466891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-600888cc-ea50-482d-9285-ebaf9cd84c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:14.503074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:14.504112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.504503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.504877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.512773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:14.514491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:14.530757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:14.531804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:14.532699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:14.533136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 411/850] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:14.539446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:14.555588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 440/851] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:14.577938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600888cc-ea50-482d-9285-ebaf9cd84c19 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:14.641599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:14.680321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493852e-11d7-48f2-b037-b2820d725fce None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:14.701796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600888cc-ea50-482d-9285-ebaf9cd84c19 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:14.716555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c34102-4e5a-4e8f-bc75-20e0b77ea252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:14.717359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c34102-4e5a-4e8f-bc75-20e0b77ea252 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.717645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c34102-4e5a-4e8f-bc75-20e0b77ea252 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.717996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c34102-4e5a-4e8f-bc75-20e0b77ea252 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.718883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c34102-4e5a-4e8f-bc75-20e0b77ea252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:14.719200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c34102-4e5a-4e8f-bc75-20e0b77ea252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:14.720636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c34102-4e5a-4e8f-bc75-20e0b77ea252 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:14.733627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 412/852] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:14.737134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:14.739818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b718d-7aa9-42da-b231-d06e629de835 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.740121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b718d-7aa9-42da-b231-d06e629de835 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.740401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b718d-7aa9-42da-b231-d06e629de835 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.745815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 413/853] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:14.745815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b718d-7aa9-42da-b231-d06e629de835 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:14.747720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac021b3-eeca-4dda-b92f-cff94028a548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:14.748838 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac021b3-eeca-4dda-b92f-cff94028a548 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.749331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac021b3-eeca-4dda-b92f-cff94028a548 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.749757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac021b3-eeca-4dda-b92f-cff94028a548 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.750578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac021b3-eeca-4dda-b92f-cff94028a548 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:14.752109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac021b3-eeca-4dda-b92f-cff94028a548 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1ac021b3-eeca-4dda-b92f-cff94028a548 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:14.765133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:14.766213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:14.767212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:14.769365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 441/854] 10.222.0.25 () {56 vars in 1132 bytes} [Fri Jun 17 11:26:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:14.770780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a1ffe-fe91-4c83-890a-ad05dbd6e94e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.771059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a1ffe-fe91-4c83-890a-ad05dbd6e94e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.771332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a1ffe-fe91-4c83-890a-ad05dbd6e94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.790244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c52a1ffe-fe91-4c83-890a-ad05dbd6e94e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:14.792158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c52a1ffe-fe91-4c83-890a-ad05dbd6e94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:14.857112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b718d-7aa9-42da-b231-d06e629de835 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:14.871484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 442/855] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:14.872413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a1ffe-fe91-4c83-890a-ad05dbd6e94e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:14.905962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:14.913061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:14.913741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:14.914038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:14.914311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:14.915137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=fd3119ad342840ddb0258e124be55597) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:14.915446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:14.917360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:14.925898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 414/856] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:14 2022] GET /identity/v3/role_assignments?user.id=fd3119ad342840ddb0258e124be55597&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:14.926919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:15.013188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a1ffe-fe91-4c83-890a-ad05dbd6e94e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:15.038709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0595da-7a7d-4ccc-b245-a56d5c42dee5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:15.053444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:15.064797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:15.066719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.067228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.067962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.069748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:15.070327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:15.073465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:15.077232 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:15.078102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.078393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.078669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.084804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 443/857] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:15.086620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:15.096721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:15.104747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:15.105779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:26:15.106664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:15.111319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 415/858] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:26:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:15.111842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:15.173543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:15.223705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504a11c-c7bf-444a-a854-8dfd74305a4a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:15.228152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21553b-12aa-4132-9c4e-cf565c9b3098 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:15.262468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:15.263181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.263425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.263702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.264557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:15.264959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:15.266453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:15.267291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:15.268149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.268499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.268882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.269596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:15.269922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:15.271282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:15.275045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 416/859] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:26:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:15.276879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 444/860] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:15.282946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:15.397833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47b0a1-7a59-4685-983e-b7815ef772ec None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:15.417081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:15.437277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:15.438134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.438556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.438834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.451582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:15.453209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:15.471996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:15.473149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:15.474615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:15.474615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:15.474615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:15.474615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:15.474615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 417/861] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:15.479067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:15.670359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:15.713096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f634a4d-b24a-4bab-b7fd-d083e474e911 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:15.756344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d5677-d692-46c8-9d85-9fa752ca97a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:15.757038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5677-d692-46c8-9d85-9fa752ca97a3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.757329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5677-d692-46c8-9d85-9fa752ca97a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.759939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d5677-d692-46c8-9d85-9fa752ca97a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.760627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d5677-d692-46c8-9d85-9fa752ca97a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:15.763412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d5677-d692-46c8-9d85-9fa752ca97a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:15.764825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d5677-d692-46c8-9d85-9fa752ca97a3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:15.777974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 445/862] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:15.784872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:15.897629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186ff4d-81cd-44c3-8307-11e85c36fc5a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.897629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186ff4d-81cd-44c3-8307-11e85c36fc5a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.897629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186ff4d-81cd-44c3-8307-11e85c36fc5a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.897629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 446/863] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:15.901548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a2ab-c2e5-4173-95e7-ad39588a1fac None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.901922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a2ab-c2e5-4173-95e7-ad39588a1fac None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.902272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a2ab-c2e5-4173-95e7-ad39588a1fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.906269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228f7fc-46dc-40fc-964d-18dea218e953 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:15.919951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-28d6a2ab-c2e5-4173-95e7-ad39588a1fac None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:15.921469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-28d6a2ab-c2e5-4173-95e7-ad39588a1fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:15.944590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:15.945456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:15.945795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:15.946117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:15.954845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:15.956606 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:15.972772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:15.973699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:15.974658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:15.975183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 418/864] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:26:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:15.981876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:16.014215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:16.022269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 447/865] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:16.044493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a2ab-c2e5-4173-95e7-ad39588a1fac None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:16.095402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:16.100899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b1529-9ab3-49f2-9b56-7ebc8f9ab24a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:16.148204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:16.149091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:16.149204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:16.149423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:16.150438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:16.151235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:16.152714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:16.173933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a2ab-c2e5-4173-95e7-ad39588a1fac None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:16.176469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 419/866] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Jun 17 11:26:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:16.177483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:16.217615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:16.218328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:16.218603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:16.218989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:16.219722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:16.221095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:16.246331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 448/867] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:16 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:26:16.288459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:16.304911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322a81c-ca83-402c-ac7f-31f5c9576fc2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:16.350463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:16.351930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:16.352447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:16.352940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:16.428730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:16.430157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:16.528152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:16.541687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 420/868] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:16 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:16.547233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:16.651703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a60c5-31eb-4d69-8628-5a771ebc052d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:16.654131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:16.687664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:16.688389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:16.688670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:16.688945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:16.700087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:16.701604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:16.719642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:16.720567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:16.721485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:16.721485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:16.721485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:16.721485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:16.721485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 449/869] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:16.726747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:16.829241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc6757-427b-44bf-873f-cd8b0c72bc4d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:16.863642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:16.864360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:16.864652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:16.864932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:16.865658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:16.866003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:16.867370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:16.879801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 421/870] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:16.885864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:16.990256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6bd5-7439-48ed-b6b1-f400a4120f8d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:17.026062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:17.026858 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.027187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.027468 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.034729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:17.036513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:17.050519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:17.053476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:17.057584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 450/871] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:17.062977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:17.115668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:17.172235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ddc65-7b2b-4017-b7ac-a4f1ce1f461c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:17.207086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389b942-3518-493c-a9b4-c93d2618df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:17.207789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b942-3518-493c-a9b4-c93d2618df46 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.208076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b942-3518-493c-a9b4-c93d2618df46 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.208354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389b942-3518-493c-a9b4-c93d2618df46 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.209171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389b942-3518-493c-a9b4-c93d2618df46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:17.209488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389b942-3518-493c-a9b4-c93d2618df46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:17.211005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389b942-3518-493c-a9b4-c93d2618df46 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:17.225411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 422/872] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:17.235027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:17.256668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:17.282783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3177-c1ca-4bf3-8229-60611c96f318 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.283070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3177-c1ca-4bf3-8229-60611c96f318 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.283354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce3177-c1ca-4bf3-8229-60611c96f318 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.285034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 423/873] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:17.290078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62b18f-d468-45e1-b4c8-52decbf567c2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.290368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62b18f-d468-45e1-b4c8-52decbf567c2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.290645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62b18f-d468-45e1-b4c8-52decbf567c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.309211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6f62b18f-d468-45e1-b4c8-52decbf567c2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:17.310444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6f62b18f-d468-45e1-b4c8-52decbf567c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:17.343192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5abd9-a284-4255-8d9a-0d40fe01b91e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:17.381995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 424/874] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:17.387732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307aa73-c278-447e-9011-66e3b2b6e553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:17.388519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307aa73-c278-447e-9011-66e3b2b6e553 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.388871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307aa73-c278-447e-9011-66e3b2b6e553 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.389212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307aa73-c278-447e-9011-66e3b2b6e553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.390199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307aa73-c278-447e-9011-66e3b2b6e553 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=84e3617bb236496783475fad6693d9af) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:17.390580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307aa73-c278-447e-9011-66e3b2b6e553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:17.392113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307aa73-c278-447e-9011-66e3b2b6e553 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:17.400862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 451/875] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:17 2022] GET /identity/v3/role_assignments?user.id=84e3617bb236496783475fad6693d9af&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:17.406973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62b18f-d468-45e1-b4c8-52decbf567c2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:17.454833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:17.534968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62b18f-d468-45e1-b4c8-52decbf567c2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:17.574434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:17.575625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.576108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.576576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.577449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:17.579043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:17.581978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec372808-98b9-454d-be29-3d19025b4166 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.582279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec372808-98b9-454d-be29-3d19025b4166 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.582556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec372808-98b9-454d-be29-3d19025b4166 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.584199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 452/876] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:17.589803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958922-aa49-49e4-962d-2e77f29dfb69 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.590133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958922-aa49-49e4-962d-2e77f29dfb69 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.590415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958922-aa49-49e4-962d-2e77f29dfb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:17.594775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:17.595944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:17.596995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:17.599509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 425/877] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:17.605954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:17.618751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d9958922-aa49-49e4-962d-2e77f29dfb69 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:17.620708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d9958922-aa49-49e4-962d-2e77f29dfb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:17.705638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 453/878] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:17.724861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:17.729531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe898b2-8227-46f0-967a-c4427eacfaf4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:17.731006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9958922-aa49-49e4-962d-2e77f29dfb69 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:17.754296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:17.770273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:17.771271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.771756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.772259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.773239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:17.773789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:17.775373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:17.780789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 426/879] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:17.786915 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:17.862992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9958922-aa49-49e4-962d-2e77f29dfb69 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:17.903447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e301-963e-4242-817f-065e806d0b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:17.904484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e301-963e-4242-817f-065e806d0b50 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.904988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e301-963e-4242-817f-065e806d0b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.905482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e301-963e-4242-817f-065e806d0b50 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.906490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e301-963e-4242-817f-065e806d0b50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:17.908046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e301-963e-4242-817f-065e806d0b50 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:17.908224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13e255-7cf3-4d16-8cc9-0ecf55616c81 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:17.923927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 454/880] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:17 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:26:17.954209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708fbbf-b91e-4211-a97f-8255dacd2344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:17.954896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708fbbf-b91e-4211-a97f-8255dacd2344 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:17.955177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708fbbf-b91e-4211-a97f-8255dacd2344 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:17.955451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708fbbf-b91e-4211-a97f-8255dacd2344 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:17.956173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708fbbf-b91e-4211-a97f-8255dacd2344 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:17.956476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708fbbf-b91e-4211-a97f-8255dacd2344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:17.957897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708fbbf-b91e-4211-a97f-8255dacd2344 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:17.963721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 427/881] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:18.537007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf4c5-a406-490d-98f6-9134ef2be181 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:18.537551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf4c5-a406-490d-98f6-9134ef2be181 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:18.537551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf4c5-a406-490d-98f6-9134ef2be181 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:18.539118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 455/882] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:18.545041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bb634-c5c7-440f-94f3-64c14e167621 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:18.545854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bb634-c5c7-440f-94f3-64c14e167621 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:18.546398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bb634-c5c7-440f-94f3-64c14e167621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:18.563361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-114bb634-c5c7-440f-94f3-64c14e167621 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:18.564824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-114bb634-c5c7-440f-94f3-64c14e167621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:18.581516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:18.636745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 428/883] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:18.659868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf4c5-a406-490d-98f6-9134ef2be181 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:18.739360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:18.777326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf4c5-a406-490d-98f6-9134ef2be181 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:18.811991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:18.812764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:18.813054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:18.813329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:18.814067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:18.815426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:18.827779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:18.828758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:18.829579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 456/884] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:18.834936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bb634-c5c7-440f-94f3-64c14e167621 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:18.937994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bb634-c5c7-440f-94f3-64c14e167621 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:18.972296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:18.972956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:18.973243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:18.973515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:18.974246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:18.974550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:18.975905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:18.980906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 429/885] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:18.986559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:19.102131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f55e5c-eb80-4d2e-84e3-a611c899851b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:19.147754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:19.148442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:19.148727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:19.149004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:19.149715 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:19.150066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:19.151471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:19.152790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:19.157403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 457/886] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:19.163624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:19.182946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:19.270157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff058d-66be-4ad6-b550-7f5bb16e4052 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:19.310130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e2df3-b346-427b-ba16-71e1b7cca109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:19.310858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e2df3-b346-427b-ba16-71e1b7cca109 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:19.311140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e2df3-b346-427b-ba16-71e1b7cca109 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:19.311415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e2df3-b346-427b-ba16-71e1b7cca109 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:19.312258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e2df3-b346-427b-ba16-71e1b7cca109 None admin] RBAC: Adding query filter params (interface=public, service_id=b0dcbfa9bbf14dddb9b1793affbaadce, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:19.312572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e2df3-b346-427b-ba16-71e1b7cca109 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:19.313962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e2df3-b346-427b-ba16-71e1b7cca109 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:19.319510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 430/887] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:26:19 2022] GET /identity/v3/endpoints?service_id=b0dcbfa9bbf14dddb9b1793affbaadce&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:19.325529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:19.428576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5d9c4-b7db-4d95-ba74-1d964cb1665b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:19.462623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322dc194-ada6-4a16-881b-782148c56280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:19.463323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dc194-ada6-4a16-881b-782148c56280 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:19.463607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dc194-ada6-4a16-881b-782148c56280 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:19.463884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dc194-ada6-4a16-881b-782148c56280 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:19.464561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322dc194-ada6-4a16-881b-782148c56280 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:19.465959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322dc194-ada6-4a16-881b-782148c56280 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:19.471944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 458/888] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:19 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:26:19.778963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767967-dd13-4944-a3ed-2111bbd17491 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:19.779485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767967-dd13-4944-a3ed-2111bbd17491 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:19.779485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767967-dd13-4944-a3ed-2111bbd17491 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:19.781144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 431/889] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:19.787371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6534c7-4c7a-406c-a7ce-fd91b14cbc2f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:19.788132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6534c7-4c7a-406c-a7ce-fd91b14cbc2f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:19.788550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6534c7-4c7a-406c-a7ce-fd91b14cbc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:19.806258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4f6534c7-4c7a-406c-a7ce-fd91b14cbc2f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:19.807725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4f6534c7-4c7a-406c-a7ce-fd91b14cbc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:19.867765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:19.885937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 459/890] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:19.905523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767967-dd13-4944-a3ed-2111bbd17491 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.002208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:20.012642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767967-dd13-4944-a3ed-2111bbd17491 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.048782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5dc265-5ebe-455b-9823-872f64486b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.049564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dc265-5ebe-455b-9823-872f64486b84 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.049864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dc265-5ebe-455b-9823-872f64486b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.050149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5dc265-5ebe-455b-9823-872f64486b84 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.061180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5dc265-5ebe-455b-9823-872f64486b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5c5dc265-5ebe-455b-9823-872f64486b84 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:20.069404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:20.070344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:20.071225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:20.075949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 432/891] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Jun 17 11:26:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:20.078027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6534c7-4c7a-406c-a7ce-fd91b14cbc2f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.190193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6534c7-4c7a-406c-a7ce-fd91b14cbc2f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.214187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3e23-090a-4454-a2d8-26484e5a98c1 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.214507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3e23-090a-4454-a2d8-26484e5a98c1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.214735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3e23-090a-4454-a2d8-26484e5a98c1 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.217495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 433/892] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:20.221947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f4be1-967d-44b4-957d-38d71f75658a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.222241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f4be1-967d-44b4-957d-38d71f75658a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.222621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f4be1-967d-44b4-957d-38d71f75658a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.232128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875f5ce-03f2-46e6-853d-9fcbb2e614e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.232813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f5ce-03f2-46e6-853d-9fcbb2e614e2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.233098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f5ce-03f2-46e6-853d-9fcbb2e614e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.233373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f5ce-03f2-46e6-853d-9fcbb2e614e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.234161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875f5ce-03f2-46e6-853d-9fcbb2e614e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:20.234470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875f5ce-03f2-46e6-853d-9fcbb2e614e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.235874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875f5ce-03f2-46e6-853d-9fcbb2e614e2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:20.240016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9b7f4be1-967d-44b4-957d-38d71f75658a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:20.241195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9b7f4be1-967d-44b4-957d-38d71f75658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:20.244903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 460/893] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:20.246372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fdcac-f845-4c2c-98f3-96d98f92fdb4 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.246663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fdcac-f845-4c2c-98f3-96d98f92fdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.246941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fdcac-f845-4c2c-98f3-96d98f92fdb4 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.248308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 461/894] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:20.252763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fdcac-f845-4c2c-98f3-96d98f92fdb4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.311978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:20.320793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 434/895] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:20.322211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46aff7-4d6e-44f7-ae25-572d5d8fe76a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.322502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46aff7-4d6e-44f7-ae25-572d5d8fe76a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.322776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46aff7-4d6e-44f7-ae25-572d5d8fe76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.347561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2d46aff7-4d6e-44f7-ae25-572d5d8fe76a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:20.349149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2d46aff7-4d6e-44f7-ae25-572d5d8fe76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:20.361481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fdcac-f845-4c2c-98f3-96d98f92fdb4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.398658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.399448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.399732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.399997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.412321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.413908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:20.418426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:20.433861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 435/896] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:20.433861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46aff7-4d6e-44f7-ae25-572d5d8fe76a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:20.436510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:20.437383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:20.438304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:20.438304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:20.438304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:20.438304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:20.438304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 462/897] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:20.443652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.545677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46aff7-4d6e-44f7-ae25-572d5d8fe76a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.558238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1bf14-7be5-4462-a90a-e48961ce05b9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.566062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:20.591101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.591918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.592263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.592540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.593191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.594576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:20.594946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.595560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.595840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.596123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.596825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:20.597121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.598570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:20.607717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:20.608736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:20.609662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 436/898] 10.222.0.25 () {56 vars in 1132 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:20.610645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.617300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 463/899] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:20.618917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.684940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:20.725759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edd1d7-fd46-41ab-a379-a32b92d8e23b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.734163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f4805-b214-454a-81ee-1aac764f2d80 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.762748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.763416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.763712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.763988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.764707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.766103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:20.769667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.770380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.770758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.770909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.771845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:20.772211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.773855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:20.779191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 464/900] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:20.783984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.784103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 437/901] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:20 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:20.784882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.832946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:20.906426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b905a-e5aa-48e9-9c8f-f2cfb9719fdc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.922581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95043d-7a0d-4ca1-a49b-7930dd8dc2ee None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:20.928896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:20.946696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.947454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.947737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.948010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.948882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:20.949244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.950639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:20.956093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 438/902] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:20.961379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:20.970818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:20.972246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:20.972537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:20.972808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:20.981343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:20.983171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:21.001567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:21.002610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:21.003982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 465/903] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:21.011127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:21.087375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd179-3f3e-4942-b9c0-8cddb9fde235 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:21.124719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:21.125394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.125682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.126019 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.126951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] RBAC: Adding query filter params (interface=public, service_id=2c0e43d4c8f44607aca84b679b5f68db, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:21.127272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:21.128649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:21.131875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:21.134745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 439/904] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:26:20 2022] GET /identity/v3/endpoints?service_id=2c0e43d4c8f44607aca84b679b5f68db&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:21.140160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:21.143498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fe9da-4839-4a1b-9d72-616247c0d814 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:21.192339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:21.193613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.194140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.194739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:21.195207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.197214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:21.197796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:21.201261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:21.232244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 466/905] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Jun 17 11:26:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:21.233941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:21.263176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa3287-abf8-470d-aa2d-1341762b68cf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:21.317094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:21.318133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.318607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.318882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.319569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:21.320944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:21.330485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 440/906] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:21 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:26:21.355083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9fb31-5eab-4a9d-bf52-91771ffca181 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:21.395299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b343f5e-35c1-44a7-9e23-f09951a916eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:21.395978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343f5e-35c1-44a7-9e23-f09951a916eb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.396257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343f5e-35c1-44a7-9e23-f09951a916eb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.396529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b343f5e-35c1-44a7-9e23-f09951a916eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.418513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:21.474076 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b343f5e-35c1-44a7-9e23-f09951a916eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:21.475546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b343f5e-35c1-44a7-9e23-f09951a916eb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:21.586032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 467/907] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:21 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:26:21.588399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:21.679030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0813e-b091-4376-9acb-e0b3fd0297e9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.679366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0813e-b091-4376-9acb-e0b3fd0297e9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.679594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0813e-b091-4376-9acb-e0b3fd0297e9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.681224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 468/908] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:21.686139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e66a-5107-4846-aac5-dfdf46408ca1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.686429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e66a-5107-4846-aac5-dfdf46408ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.686708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375e66a-5107-4846-aac5-dfdf46408ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.701722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1fcdb-d34c-4cdd-ac0f-bb3eba6007e6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:21.706326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7375e66a-5107-4846-aac5-dfdf46408ca1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:21.707530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7375e66a-5107-4846-aac5-dfdf46408ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:21.715416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:21.740883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:21.741681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.741998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.742269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.755257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:21.756690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:21.774246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:21.780702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:21.781649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:21.782613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:21.782613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:21.782613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:21.782613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:21.787696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 441/909] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:21.789088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:21.821973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 469/910] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:21.839332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e66a-5107-4846-aac5-dfdf46408ca1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:21.898671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d24994-c250-4476-a5ce-9ef7844fec5f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:21.935182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:21.936052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.936337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.936612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.937331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:21.937634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:21.939043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:21.952381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375e66a-5107-4846-aac5-dfdf46408ca1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:21.956133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 442/911] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:21.957395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:21.985018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:21.994392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:21.995260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:21.995545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:21.995865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:21.996629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:21.998132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:22.011496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:22.012733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 17 11:26:22.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:22.017697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 470/912] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:22.018682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:22.065663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:22.075811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b2a64-c245-4efb-b323-34f312010780 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:22.123023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:22.124401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:22.125079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:22.125974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:22.131774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2a59c-dca3-42d6-9d43-d9f84d749172 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:22.136462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:22.138153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:22.157146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:22.158101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:22.158989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:22.164840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 443/913] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:22.166554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:22.176197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:22.176879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:22.177166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:22.177446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:22.178208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:22.178520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:22.179890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:22.190095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 471/914] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:22.191892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:22.298996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8bead-5d5b-49eb-9b67-5e1256b32fb8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:22.300101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7405eb-acd8-408e-a6c4-4bdc5f275f3e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:22.302164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:22.310392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:22.334903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:22.335740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:22.336031 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:22.336318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:22.337116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:22.337430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:22.339039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:22.347176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:22.348188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:22.349067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:22.349588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:22.350785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:22.351284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:22.357018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 444/915] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:22.357174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:22.359208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:22.366098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 472/916] 10.222.0.25 () {56 vars in 1114 bytes} [Fri Jun 17 11:26:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:22.367042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:22.481336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89e997-333c-4d6d-909a-11d8857bbd75 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:22.486579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ad80-0d83-4672-86ce-3f468eda1032 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:22.530800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419425a7-4968-4953-931a-ebbfc405ca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:22.531573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419425a7-4968-4953-931a-ebbfc405ca90 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:22.531859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419425a7-4968-4953-931a-ebbfc405ca90 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:22.532097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a463ea-cde7-406d-a4b0-3f0bf54488af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:22.532207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419425a7-4968-4953-931a-ebbfc405ca90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:22.532978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a463ea-cde7-406d-a4b0-3f0bf54488af None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:22.533096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419425a7-4968-4953-931a-ebbfc405ca90 None admin] RBAC: Adding query filter params (role.id=d4c16bd9276449d887ea030f63a2cde1, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=84e3617bb236496783475fad6693d9af) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:22.533370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419425a7-4968-4953-931a-ebbfc405ca90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:22.533625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a463ea-cde7-406d-a4b0-3f0bf54488af None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:22.534070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a463ea-cde7-406d-a4b0-3f0bf54488af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:22.534827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419425a7-4968-4953-931a-ebbfc405ca90 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:22.535270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a463ea-cde7-406d-a4b0-3f0bf54488af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:22.537412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a463ea-cde7-406d-a4b0-3f0bf54488af None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:22.549882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 445/917] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:22 2022] GET /identity/v3/role_assignments?user.id=84e3617bb236496783475fad6693d9af&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=d4c16bd9276449d887ea030f63a2cde1 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:22.566992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 473/918] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:23.065617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efcb51-3b9e-480c-8377-a4b42e39032d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.066191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efcb51-3b9e-480c-8377-a4b42e39032d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.066191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efcb51-3b9e-480c-8377-a4b42e39032d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.067783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 446/919] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:23.073278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b878f-6a73-4dca-bd10-1be43159253d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.073717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b878f-6a73-4dca-bd10-1be43159253d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.073925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b878f-6a73-4dca-bd10-1be43159253d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.091848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-737b878f-6a73-4dca-bd10-1be43159253d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:23.093044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-737b878f-6a73-4dca-bd10-1be43159253d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:23.117511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:23.173702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 474/920] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:23.195867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efcb51-3b9e-480c-8377-a4b42e39032d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:23.208173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:23.306160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efcb51-3b9e-480c-8377-a4b42e39032d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:23.343305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:23.344154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.344443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.344719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.345376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:23.346785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:23.359745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:23.360768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:23.362071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 447/921] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:23.367490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b878f-6a73-4dca-bd10-1be43159253d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:23.473993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b878f-6a73-4dca-bd10-1be43159253d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:23.508236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a356c-97f4-40ab-a8ea-285f0391bb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:23.508949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a356c-97f4-40ab-a8ea-285f0391bb37 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.509239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a356c-97f4-40ab-a8ea-285f0391bb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.510083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a356c-97f4-40ab-a8ea-285f0391bb37 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.510832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a356c-97f4-40ab-a8ea-285f0391bb37 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:23.511148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a356c-97f4-40ab-a8ea-285f0391bb37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:23.512652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a356c-97f4-40ab-a8ea-285f0391bb37 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:23.518736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 475/922] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:23.524304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:23.579180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34694-6f8d-4055-b686-3092b3fefb4b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.579477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34694-6f8d-4055-b686-3092b3fefb4b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.579736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb34694-6f8d-4055-b686-3092b3fefb4b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.581367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 476/923] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:23.586094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccc9a-33a3-463c-87c6-8b104116c272 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.586382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccc9a-33a3-463c-87c6-8b104116c272 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.586661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccc9a-33a3-463c-87c6-8b104116c272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.602014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:23.603217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-85fccc9a-33a3-463c-87c6-8b104116c272 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:23.604437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-85fccc9a-33a3-463c-87c6-8b104116c272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:23.616095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:23.650749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9b4de-2a74-468f-8bbe-e2d22427d0a8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:23.682600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 477/924] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:23.690290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:23.691294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.691784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.692260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.693198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:23.693700 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:23.695343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:23.702018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccc9a-33a3-463c-87c6-8b104116c272 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:23.702821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 448/925] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:23.709092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:23.817342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccc9a-33a3-463c-87c6-8b104116c272 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:23.825497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f40d7-4c52-4b29-a913-61ba681d5014 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:23.857803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:23.858954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.859421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.859928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.860874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:23.862575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:23.865638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:23.866450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:23.866809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:23.867084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:23.868063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] RBAC: Adding query filter params (interface=public, service_id=10500353e2fd434dad1dc75a7d341ad5, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:23.868454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:23.870498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:23.873031 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:23.877905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 449/926] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:26:23 2022] GET /identity/v3/endpoints?service_id=10500353e2fd434dad1dc75a7d341ad5&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:23.883096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:23.884058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:23.886124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 478/927] 10.222.0.25 () {56 vars in 1132 bytes} [Fri Jun 17 11:26:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:23.886982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:23.890420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:23.906054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:24.007343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5f573-6807-4a66-ba0c-d9156d6c6307 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:24.017404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd07bac-4df0-4496-846a-7e21a9c9bd01 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:24.043677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:24.044643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.045123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.045638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.046654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:24.048209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:24.054266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:24.055010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.055216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.055632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.055874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 450/928] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:23 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:26:24.056200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:24.057092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:24.062750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:24.071027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 479/929] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:24.079924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:24.150384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:24.189360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19690ae-5b28-404e-b975-d2b305b427c1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:24.225882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180cf20-bc3b-483c-9f60-4c7f8ead8f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:24.226913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180cf20-bc3b-483c-9f60-4c7f8ead8f0b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.227419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180cf20-bc3b-483c-9f60-4c7f8ead8f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.227893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180cf20-bc3b-483c-9f60-4c7f8ead8f0b None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.229348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180cf20-bc3b-483c-9f60-4c7f8ead8f0b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:24.229863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180cf20-bc3b-483c-9f60-4c7f8ead8f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:24.231419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180cf20-bc3b-483c-9f60-4c7f8ead8f0b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:24.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 451/930] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:24.243031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:24.324594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:24.351397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1d356-dcc5-48b2-a362-004fc25603c3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:24.388292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd6132-861d-45a7-be80-79447a9dc7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:24.388952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd6132-861d-45a7-be80-79447a9dc7cf None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.389242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd6132-861d-45a7-be80-79447a9dc7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.389523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd6132-861d-45a7-be80-79447a9dc7cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.390318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd6132-861d-45a7-be80-79447a9dc7cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:24.391686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd6132-861d-45a7-be80-79447a9dc7cf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:24.413544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 480/931] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:24.797134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a9155-06e9-4dee-a6d1-9aaf059cc810 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.797517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a9155-06e9-4dee-a6d1-9aaf059cc810 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.797711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a9155-06e9-4dee-a6d1-9aaf059cc810 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.799233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 452/932] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:24.804581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52304a9-25d7-424d-b272-bb45ef85306b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.804885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52304a9-25d7-424d-b272-bb45ef85306b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.805166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52304a9-25d7-424d-b272-bb45ef85306b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.806376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b5e47-f068-4a41-a085-fe0319fcc05e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.807299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b5e47-f068-4a41-a085-fe0319fcc05e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.807581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b5e47-f068-4a41-a085-fe0319fcc05e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.809133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 453/933] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:24.814341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c3720-55af-431a-b9d6-ca1b574c6201 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:24.814629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c3720-55af-431a-b9d6-ca1b574c6201 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:24.814907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c3720-55af-431a-b9d6-ca1b574c6201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:24.823257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f52304a9-25d7-424d-b272-bb45ef85306b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:24.824466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f52304a9-25d7-424d-b272-bb45ef85306b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:24.831486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c85c3720-55af-431a-b9d6-ca1b574c6201 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:24.832770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c85c3720-55af-431a-b9d6-ca1b574c6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:24.909544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 481/934] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:24.920678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 454/935] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:24.934757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52304a9-25d7-424d-b272-bb45ef85306b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:24.955527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c3720-55af-431a-b9d6-ca1b574c6201 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:25.003986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:25.029017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:25.068642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52304a9-25d7-424d-b272-bb45ef85306b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.076180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c3720-55af-431a-b9d6-ca1b574c6201 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.112905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.113853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.114234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.114339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.114594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.115425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.115874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.116321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.117376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:25.119427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:25.121467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING keystone.server.flask.common [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 17 11:26:25.125965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:25.134470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:25.135433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:25.141484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 455/936] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Jun 17 11:26:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:25.144199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:25.147141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.sql.core [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 11:26:25.147141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 11:26:25.147141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [parameters: {'id': '9dd1353b13594000a0d89d3c78c435b4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 11:26:25.147141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80103) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9dd1353b13594000a0d89d3c78c435b4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9dd1353b13594000a0d89d3c78c435b4. Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:26:25.158146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 17 11:26:25.158982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application yield resource Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise self.value Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application session.commit() Jun 17 11:26:25.159962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.flush() Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application compat.raise_( Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 11:26:25.160865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 11:26:25.162909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:26:25.163684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [parameters: {'id': '9dd1353b13594000a0d89d3c78c435b4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.164541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:25.165331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9dd1353b13594000a0d89d3c78c435b4. Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.167317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 482/937] 10.222.0.25 () {60 vars in 1140 bytes} [Fri Jun 17 11:26:24 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:25.173253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:25.265593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac747be-d0e9-4514-9b68-615c4ddb38d5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.284169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:25.286551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd224-94f9-43c7-8b0a-60d8338d16d9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.303904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:25.305129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.306166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.306796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.307456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.308745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:25.309256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:25.311409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:25.327892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 456/938] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Jun 17 11:26:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:25.329417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:25.335937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.336796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.337085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.337363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.346551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:25.348519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:25.368086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:25.369036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:25.370353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.373917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 483/939] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:25.375682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:25.469067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902cef8-86a8-4e22-974b-f00c089ec4fe None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.493636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e987f4b-53ee-4bb6-9f0b-49f102ea3930 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.515521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.516603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.517081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.518102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.533549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:25.533795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.534544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.534833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.535114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.536345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:25.536661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:25.536926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:25.538091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:25.558474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:25.559449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:25.560318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:25.560318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:25.560318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:25.560318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:25.567145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 457/940] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:25.567903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 484/941] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:25 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:25.569124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:25.605885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:25.693087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe50ef4-1345-44f0-a02b-bf6d44e3ce5c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.732298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.733038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.733326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.733599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.734359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:25.734662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:25.736104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:25.752130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 458/942] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:25.755434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:25.791742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:25.869758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29830b-2870-464a-929b-88ae12ffb1b1 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:25.905508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:25.906357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:25.906643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:25.906920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:25.914670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:25.916139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:25.932025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:25.932985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:25.933937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:25.935005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 485/943] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:26:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:25.941043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.038817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:26.050027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb672d-6b58-4119-b575-743a3589bdd4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.091743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0719c4a-c591-475a-b6ac-52d3ed909c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.092789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719c4a-c591-475a-b6ac-52d3ed909c83 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.093268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719c4a-c591-475a-b6ac-52d3ed909c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.093781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0719c4a-c591-475a-b6ac-52d3ed909c83 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.094847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0719c4a-c591-475a-b6ac-52d3ed909c83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:26.095351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0719c4a-c591-475a-b6ac-52d3ed909c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.096981 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0719c4a-c591-475a-b6ac-52d3ed909c83 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.121276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 459/944] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Jun 17 11:26:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:26.123473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.226775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:26.240450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483480e-7cac-495d-8ef3-e5bc6e05f92a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.280274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.281465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.281973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.282442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.294073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.296141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.303188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a976e42-ce7d-4323-afa3-45af532effa2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.303480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a976e42-ce7d-4323-afa3-45af532effa2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.303753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a976e42-ce7d-4323-afa3-45af532effa2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.305268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 460/945] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:26.310850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb149-4100-4590-9e9b-d722ad352b48 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.311377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb149-4100-4590-9e9b-d722ad352b48 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.311878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cb149-4100-4590-9e9b-d722ad352b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:26.320682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:26.321594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:26.322544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:26.322544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:26.322544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:26.322544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:26.325924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 486/946] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:26.329319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.334872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5c8cb149-4100-4590-9e9b-d722ad352b48 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:26.336305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5c8cb149-4100-4590-9e9b-d722ad352b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:26.430339 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 461/947] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:26.437566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613d629-39c9-44d1-801d-59891803b567 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.454795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb149-4100-4590-9e9b-d722ad352b48 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.480914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.482210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.482935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.483840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.485059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:26.485814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.488093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.509083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 487/948] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:26.511122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.515920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:26.548470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:26.577465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cb149-4100-4590-9e9b-d722ad352b48 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.616493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff195087-a89e-4121-9429-a52e872d3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.617498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff195087-a89e-4121-9429-a52e872d3b08 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.617782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff195087-a89e-4121-9429-a52e872d3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.618086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff195087-a89e-4121-9429-a52e872d3b08 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.618822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff195087-a89e-4121-9429-a52e872d3b08 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.620377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff195087-a89e-4121-9429-a52e872d3b08 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ff195087-a89e-4121-9429-a52e872d3b08 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:26.632660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:26.633607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:26.634531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:26.639036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 462/949] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:26.640352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0e880-c625-4833-b780-4032dfdb3cb0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.640648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0e880-c625-4833-b780-4032dfdb3cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.640928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0e880-c625-4833-b780-4032dfdb3cb0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.645210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 463/950] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:26.645210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0e880-c625-4833-b780-4032dfdb3cb0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.645424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedbd28-b371-4274-9cfb-7e5d0b0ee548 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.688071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f89eeb-7df8-434a-9c47-6988334fc814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.689643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f89eeb-7df8-434a-9c47-6988334fc814 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.690104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f89eeb-7df8-434a-9c47-6988334fc814 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.690520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f89eeb-7df8-434a-9c47-6988334fc814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.699490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f89eeb-7df8-434a-9c47-6988334fc814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.701708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f89eeb-7df8-434a-9c47-6988334fc814 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-34f89eeb-7df8-434a-9c47-6988334fc814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:26.716942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:26.717878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 488/951] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be2d4-8946-4b56-9d5b-fb437f42f3ad None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.723447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be2d4-8946-4b56-9d5b-fb437f42f3ad None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.724091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738be2d4-8946-4b56-9d5b-fb437f42f3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.738339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-738be2d4-8946-4b56-9d5b-fb437f42f3ad None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:26.739934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-738be2d4-8946-4b56-9d5b-fb437f42f3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:26.752522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0e880-c625-4833-b780-4032dfdb3cb0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.790005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.791735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.792030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.792311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.793045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:26.793356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.794785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.800637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 464/952] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:26.804363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:26.805750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.814693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 489/953] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:26.817901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:26.826741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be2d4-8946-4b56-9d5b-fb437f42f3ad None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.913603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5de7a-4e98-4064-adbf-82d950abce3e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.941766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738be2d4-8946-4b56-9d5b-fb437f42f3ad None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:26.950216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.950885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.951170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.951443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.952228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:26.952536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.953977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.966665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 465/954] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:26.967458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:26.983488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:26.984162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:26.984445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:26.984727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:26.985436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:26.985745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:26.987133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:26.992050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 490/955] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:26.992790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:27.047144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:27.054976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:27.084190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbafd9-8cfe-48e1-9b33-ceaf3a5a2d10 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:27.103085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4c3da-4138-4b6a-b541-e40cfebb702a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:27.122230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:27.123102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:27.123389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:27.123657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:27.124414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:27.125800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:27.138751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:27.145200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:27.146190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:27.146965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:27.146965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:27.146965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:27.147178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 466/956] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:27.147247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/84e3617bb236496783475fad6693d9af/roles/d4c16bd9276449d887ea030f63a2cde1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:27.147318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:27.148996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=84e3617bb236496783475fad6693d9af, role_id=d4c16bd9276449d887ea030f63a2cde1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:27.151029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:27.166257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 491/957] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:26:26 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/84e3617bb236496783475fad6693d9af/roles/d4c16bd9276449d887ea030f63a2cde1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:26:27.167064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:27.279324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c172949-94d6-4b98-89dc-150e2f512043 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:27.304642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809ce29-bcc5-438f-93e7-c25b0870672c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:27.315423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:27.322808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67938ce-6943-46ef-8bc6-3fd013bb8759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:27.324508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67938ce-6943-46ef-8bc6-3fd013bb8759 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:27.325131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67938ce-6943-46ef-8bc6-3fd013bb8759 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:27.325641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67938ce-6943-46ef-8bc6-3fd013bb8759 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:27.326642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67938ce-6943-46ef-8bc6-3fd013bb8759 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:27.327910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:27.329503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67938ce-6943-46ef-8bc6-3fd013bb8759 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:27.351162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:27.351870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:27.352060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:27.352361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:27.353075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] RBAC: Adding query filter params (name=compute) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:27.353579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:27.354906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:27.361062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 492/958] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:27.363278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 467/959] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:27.366938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:27.500062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda866e-17ed-4c54-9d8b-9cab9febbdde None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:27.549655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c758f-b327-47ba-b3c8-ab4faaf2b491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:27.550342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c758f-b327-47ba-b3c8-ab4faaf2b491 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:27.550629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c758f-b327-47ba-b3c8-ab4faaf2b491 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:27.550959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c758f-b327-47ba-b3c8-ab4faaf2b491 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:27.551668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c758f-b327-47ba-b3c8-ab4faaf2b491 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:27.551979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c758f-b327-47ba-b3c8-ab4faaf2b491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:27.553346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c758f-b327-47ba-b3c8-ab4faaf2b491 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:27.559323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 493/960] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:27.868099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aedde0-5fcf-4df4-9af3-4a579af24f28 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:27.868400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aedde0-5fcf-4df4-9af3-4a579af24f28 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:27.868596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aedde0-5fcf-4df4-9af3-4a579af24f28 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:27.870780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 468/961] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:27.876222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357224-3eb4-4f02-8154-335800ec4c31 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:27.876513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357224-3eb4-4f02-8154-335800ec4c31 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:27.876793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357224-3eb4-4f02-8154-335800ec4c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:27.894037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7a357224-3eb4-4f02-8154-335800ec4c31 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:27.895240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7a357224-3eb4-4f02-8154-335800ec4c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:27.934715 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:27.976898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 494/962] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:28.001416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aedde0-5fcf-4df4-9af3-4a579af24f28 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:28.114911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aedde0-5fcf-4df4-9af3-4a579af24f28 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:28.149328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:28.150605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:28.150935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:28.151221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:28.151947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:28.153386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:28.154940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING keystone.server.flask.common [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 17 11:26:28.168180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.sql.core [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 11:26:28.168180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 11:26:28.168180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [parameters: {'user_id': 'a2bece399d974e239e1eec2db9411a38', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 11:26:28.168180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80102) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result.read() Jun 17 11:26:28.185467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:26:28.186426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application yield resource Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application next(self.gen) Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.session.rollback() Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.force_reraise() Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise self.value Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application session.commit() Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 17 11:26:28.187374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._prepare_impl() Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.session.flush() Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._flush(objects) Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application compat.raise_( Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application flush_context.execute() Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rec.execute(self) Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 17 11:26:28.188292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 17 11:26:28.189202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result = self._query(query) Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application conn.query(q) Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result.read() Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 17 11:26:28.190081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a2bece399d974e239e1eec2db9411a38', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:28.190928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 17 11:26:28.191738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 17 11:26:28.192642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 17 11:26:28.192642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 17 11:26:28.192642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 17 11:26:28.192642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 17 11:26:28.192642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 17 11:26:28.192642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:28.192642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 469/963] 10.222.0.25 () {60 vars in 1131 bytes} [Fri Jun 17 11:26:28 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:28.199181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a357224-3eb4-4f02-8154-335800ec4c31 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:28.305005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a357224-3eb4-4f02-8154-335800ec4c31 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:28.340212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841cf4ff-2b69-4a7e-bc61-428b6423fa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:28.341113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cf4ff-2b69-4a7e-bc61-428b6423fa74 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:28.341416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cf4ff-2b69-4a7e-bc61-428b6423fa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:28.341692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841cf4ff-2b69-4a7e-bc61-428b6423fa74 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:28.349585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cf4ff-2b69-4a7e-bc61-428b6423fa74 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:28.351130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841cf4ff-2b69-4a7e-bc61-428b6423fa74 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-841cf4ff-2b69-4a7e-bc61-428b6423fa74 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:28.368787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:28.371160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:28.373426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 495/964] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:28.379170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:28.437265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:28.491699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2396e-4ffb-4210-9f25-f3a77fb54d59 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:28.530737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa506f14-37cf-4d46-9efb-72af364cca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:28.531419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa506f14-37cf-4d46-9efb-72af364cca59 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:28.531707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa506f14-37cf-4d46-9efb-72af364cca59 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:28.531985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa506f14-37cf-4d46-9efb-72af364cca59 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:28.532798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa506f14-37cf-4d46-9efb-72af364cca59 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:28.533106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa506f14-37cf-4d46-9efb-72af364cca59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:28.534780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa506f14-37cf-4d46-9efb-72af364cca59 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:28.573818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 470/965] 10.222.0.25 () {56 vars in 1101 bytes} [Fri Jun 17 11:26:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:29.438516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522ccc-3a2f-41b9-85fb-b2f32e2efb6d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.439054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522ccc-3a2f-41b9-85fb-b2f32e2efb6d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.439054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522ccc-3a2f-41b9-85fb-b2f32e2efb6d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.440488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 496/966] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:29.445796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb05de3-c1e0-44bb-9883-5134c55f79d4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.446210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb05de3-c1e0-44bb-9883-5134c55f79d4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.446396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb05de3-c1e0-44bb-9883-5134c55f79d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.463392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5fb05de3-c1e0-44bb-9883-5134c55f79d4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:29.465020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5fb05de3-c1e0-44bb-9883-5134c55f79d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:29.539292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 471/967] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:29.556324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6700-e902-4506-8d7d-225edd1bba20 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.556646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6700-e902-4506-8d7d-225edd1bba20 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.556848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6700-e902-4506-8d7d-225edd1bba20 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.558375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 497/968] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:29.562751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6700-e902-4506-8d7d-225edd1bba20 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:29.563260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24992271-07da-4f50-9548-f2795cef415c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.563561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24992271-07da-4f50-9548-f2795cef415c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.563845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24992271-07da-4f50-9548-f2795cef415c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.581694 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-24992271-07da-4f50-9548-f2795cef415c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:29.583206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-24992271-07da-4f50-9548-f2795cef415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:29.597404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:29.617240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:29.660822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 472/969] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:29.677359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6700-e902-4506-8d7d-225edd1bba20 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:29.683885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24992271-07da-4f50-9548-f2795cef415c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:29.714337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:29.715093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.715391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.715970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.727801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:29.736670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:29.737974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:29.738819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 498/970] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Jun 17 11:26:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:29.744172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:29.799486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24992271-07da-4f50-9548-f2795cef415c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:29.832449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffdaed-5d95-47f7-8801-6cc93d5057f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:29.833254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffdaed-5d95-47f7-8801-6cc93d5057f1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.833536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffdaed-5d95-47f7-8801-6cc93d5057f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.833809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffdaed-5d95-47f7-8801-6cc93d5057f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.834489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffdaed-5d95-47f7-8801-6cc93d5057f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:29.835858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffdaed-5d95-47f7-8801-6cc93d5057f1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:29.839427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b8ffdaed-5d95-47f7-8801-6cc93d5057f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:29.848134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:29.854089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:29.854978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:29.855728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf91fd-e4d5-4a35-860b-59e3888d74d9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:29.855804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 473/971] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:29.855804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280a071-6c7d-4fdf-9c3d-58be22ff4dd1 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.855804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280a071-6c7d-4fdf-9c3d-58be22ff4dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.855804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280a071-6c7d-4fdf-9c3d-58be22ff4dd1 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.855804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 474/972] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:29.857266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280a071-6c7d-4fdf-9c3d-58be22ff4dd1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:29.887047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0f91b-1c48-4157-b24d-a08bdb16edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:29.887720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f91b-1c48-4157-b24d-a08bdb16edf5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.887998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f91b-1c48-4157-b24d-a08bdb16edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.888270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f91b-1c48-4157-b24d-a08bdb16edf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.889011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0f91b-1c48-4157-b24d-a08bdb16edf5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:29.889311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0f91b-1c48-4157-b24d-a08bdb16edf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:29.890698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0f91b-1c48-4157-b24d-a08bdb16edf5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:29.896719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:29.899863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 499/973] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Jun 17 11:26:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:29.901667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aadcd3-a580-44cc-ad88-8e927b433ac9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:29.902012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aadcd3-a580-44cc-ad88-8e927b433ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:29.902295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aadcd3-a580-44cc-ad88-8e927b433ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:29.918346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d0aadcd3-a580-44cc-ad88-8e927b433ac9 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:29.919531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d0aadcd3-a580-44cc-ad88-8e927b433ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:29.978931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280a071-6c7d-4fdf-9c3d-58be22ff4dd1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:29.987742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 500/974] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:29.988581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aadcd3-a580-44cc-ad88-8e927b433ac9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.021682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.023331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.023613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.023887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.024591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:30.025447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.026839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.031832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 475/975] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:30.032593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.073784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:30.089686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aadcd3-a580-44cc-ad88-8e927b433ac9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.123011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.123797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.124080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.124350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.124946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:30.131632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1171c8c-26ee-42ae-861c-965ec63e69b2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.135670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.137116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:30.155129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:30.156069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:30.157114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:30.157114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:30.157114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:30.157114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:30.157114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 501/976] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:30.157865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.169724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.170427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.170710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.171096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.171825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.173189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.186106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 476/977] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:30 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:30.186889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.286111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f11e5-7e4d-41ec-b0ce-017acef77227 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.305966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db2fce-4e7b-4857-a9ef-7344a752c80b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.325873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.326642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.326928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.327207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.327964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:30.328365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.329764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.332431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:30.342514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 502/978] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:30.343771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.344506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.345283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.345670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.346069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.346900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:30.347311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.348771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.367762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 477/979] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:30.369559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.386882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:30.470301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e3673-aaf0-436d-a809-715d3538ea25 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.478319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01377f41-399b-480a-b19e-dd8469f17861 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.507459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.508214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.508567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.508855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.510264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] RBAC: Adding query filter params (interface=admin, service_id=10500353e2fd434dad1dc75a7d341ad5, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:30.510585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.511952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.514258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.515076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.515394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.515651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.519883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 503/980] 10.222.0.25 () {56 vars in 1249 bytes} [Fri Jun 17 11:26:30 2022] GET /identity/v3/endpoints?service_id=10500353e2fd434dad1dc75a7d341ad5&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:30.523148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.524587 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.525928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:30.540145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:30.541133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:30.543104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 478/981] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:26:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:30.546933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.595293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:30.637203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:30.643458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098edfc7-807e-4494-8e24-b4ef47ff6a80 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.668930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a66fa-ead2-4295-a249-3fd284a3cf41 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.682306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.682986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.683280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.683564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.684257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.685622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.692924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 504/982] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:30 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:26:30.706018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.706733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.707023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.707300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.708177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:30.708497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.710021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:30.735378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 479/983] 10.222.0.25 () {56 vars in 1149 bytes} [Fri Jun 17 11:26:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:30.737333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:30.848874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea156cb-817d-4453-90be-43ed1cb21a48 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:30.878893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:30.888579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:30.889240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:30.889528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:30.889806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:30.961140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:30.963311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:31.068116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 505/984] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:30 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:31.069557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:31.172867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661675c-f3e5-4859-9fe2-d0eb9a5477ff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:31.204545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:31.212949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:31.214240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:31.214535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:31.214814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:31.225692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:31.227239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:31.246353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:31.247250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:31.252758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:31.252758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:31.252758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:31.252758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:31.252758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 480/985] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:31.254678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:31.284699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:31.362701 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0322a95-56b2-4aaa-9d3b-3a51582522ba None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:31.398719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:31.399391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:31.399677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:31.399976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:31.400698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:31.401007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:31.402474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:31.419510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 506/986] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:31.420537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:31.533053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fcdd2-e9f6-4e77-9408-79a55c191eb3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:31.568162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:31.568963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:31.569249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:31.569527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:31.576144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:31.577576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:31.591116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:31.591980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:31.597561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 481/987] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:31.599646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:31.703007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:31.716236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1294458-7a12-46cc-a6c0-931351148db0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:31.751316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd914dd2-d24d-4d60-9c7e-20a18e01bf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:31.751981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd914dd2-d24d-4d60-9c7e-20a18e01bf12 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:31.752266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd914dd2-d24d-4d60-9c7e-20a18e01bf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:31.752539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd914dd2-d24d-4d60-9c7e-20a18e01bf12 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:31.753327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd914dd2-d24d-4d60-9c7e-20a18e01bf12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:31.753638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd914dd2-d24d-4d60-9c7e-20a18e01bf12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:31.755077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd914dd2-d24d-4d60-9c7e-20a18e01bf12 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:31.772706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 507/988] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:31.774182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:31.821349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:31.882337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993dae6-2ea8-42af-9c16-fe7b7706a76d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:31.924398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fe40d-9ad7-414f-a085-1ee65d8f60c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:31.925071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe40d-9ad7-414f-a085-1ee65d8f60c8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:31.925358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe40d-9ad7-414f-a085-1ee65d8f60c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:31.925639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe40d-9ad7-414f-a085-1ee65d8f60c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:31.926511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe40d-9ad7-414f-a085-1ee65d8f60c8 None admin] RBAC: Adding query filter params (role.id=d4c16bd9276449d887ea030f63a2cde1, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=84e3617bb236496783475fad6693d9af) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:31.926828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe40d-9ad7-414f-a085-1ee65d8f60c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:31.928251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe40d-9ad7-414f-a085-1ee65d8f60c8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:31.937870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 482/989] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:31 2022] GET /identity/v3/role_assignments?user.id=84e3617bb236496783475fad6693d9af&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=d4c16bd9276449d887ea030f63a2cde1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:32.328078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0693453-94dc-4958-a00f-3b98a46359ba None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:32.328475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0693453-94dc-4958-a00f-3b98a46359ba None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:32.328667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0693453-94dc-4958-a00f-3b98a46359ba None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:32.330173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 508/990] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:32.335046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fa5de-553f-4377-a41b-ddc619032bd1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:32.335343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fa5de-553f-4377-a41b-ddc619032bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:32.335632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fa5de-553f-4377-a41b-ddc619032bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:32.353542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1e0fa5de-553f-4377-a41b-ddc619032bd1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:32.354768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1e0fa5de-553f-4377-a41b-ddc619032bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:32.424055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 483/991] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:32.447200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0693453-94dc-4958-a00f-3b98a46359ba None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:32.550876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0693453-94dc-4958-a00f-3b98a46359ba None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:32.584592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:32.585305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:32.585601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:32.585892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:32.586547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:32.588564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:32.600248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:32.601625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:32.603477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:32.604666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 509/992] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:32.607914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fa5de-553f-4377-a41b-ddc619032bd1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:32.650896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:32.722819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fa5de-553f-4377-a41b-ddc619032bd1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:32.757023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4f31-1eff-483b-a7b3-5f028fb7787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:32.757712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4f31-1eff-483b-a7b3-5f028fb7787f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:32.758025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4f31-1eff-483b-a7b3-5f028fb7787f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:32.758302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4f31-1eff-483b-a7b3-5f028fb7787f None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:32.759067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4f31-1eff-483b-a7b3-5f028fb7787f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:32.759382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4f31-1eff-483b-a7b3-5f028fb7787f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:32.760793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4f31-1eff-483b-a7b3-5f028fb7787f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:32.771529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 484/993] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:32.772548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:32.790262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:32.834883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f19b-9480-43d7-aff9-85e5bda20a56 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:32.835181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f19b-9480-43d7-aff9-85e5bda20a56 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:32.835373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194f19b-9480-43d7-aff9-85e5bda20a56 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:32.836886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 485/994] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:32.842146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552335f7-3642-4dcd-ab4b-d3e5419cfd7b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:32.842437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552335f7-3642-4dcd-ab4b-d3e5419cfd7b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:32.842715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552335f7-3642-4dcd-ab4b-d3e5419cfd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:32.861251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-552335f7-3642-4dcd-ab4b-d3e5419cfd7b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:32.862561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-552335f7-3642-4dcd-ab4b-d3e5419cfd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:32.881404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba69bf-0b1c-4be8-8cf2-84788ab05ba7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:32.917346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:32.918039 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:32.918334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:32.918606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:32.919311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:32.919628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:32.920974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:32.926706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 510/995] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:32.932525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:32.937289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 486/996] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:32.960440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552335f7-3642-4dcd-ab4b-d3e5419cfd7b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:32.991671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:33.014978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:33.040908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb795df-465d-460c-bd74-67b6c24bde7a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:33.072423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552335f7-3642-4dcd-ab4b-d3e5419cfd7b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:33.077737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:33.078438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:33.078725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:33.079004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:33.079798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] RBAC: Adding query filter params (interface=public, service_id=75167985ded34e01b8e85e0d69c15875, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:33.080113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:33.081956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:33.088464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 511/997] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:26:32 2022] GET /identity/v3/endpoints?service_id=75167985ded34e01b8e85e0d69c15875&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:33.093627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:33.110679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:33.111525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:33.111836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:33.112108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:33.112761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:33.114165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:33.126845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:33.127807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 17 11:26:33.128659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:33.132952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 487/998] 10.222.0.25 () {56 vars in 1126 bytes} [Fri Jun 17 11:26:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:33.134846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:33.203815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf192d-abdb-47b5-a961-0cf9805d12f3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:33.238994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:33.239673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:33.239963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:33.240241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:33.240925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:33.242445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:33.245799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:33.253884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 512/999] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:33 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:26:33.265451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad87ca-d9b6-4c85-8e17-bc127723b606 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:33.303641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:33.304342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:33.304631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:33.304910 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:33.305640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:33.306578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:33.307949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:33.318295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 488/1000] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:33.319995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:33.325731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:33.433532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e1750-cdbc-4c50-bc5f-662ca92c80f5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:33.467660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd1367-cbcb-49c7-aea1-bc0e7d506574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:33.468401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1367-cbcb-49c7-aea1-bc0e7d506574 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:33.468682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1367-cbcb-49c7-aea1-bc0e7d506574 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:33.468954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1367-cbcb-49c7-aea1-bc0e7d506574 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:33.469715 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd1367-cbcb-49c7-aea1-bc0e7d506574 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:33.470057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd1367-cbcb-49c7-aea1-bc0e7d506574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:33.471416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd1367-cbcb-49c7-aea1-bc0e7d506574 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:33.476636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 513/1001] 10.222.0.25 () {56 vars in 1122 bytes} [Fri Jun 17 11:26:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:33.482309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:33.587299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24fb0f5-d246-482a-8aef-4d5d958224b6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:33.625965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4514d40-1a22-4034-8b19-d9da093cda37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:33.626713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4514d40-1a22-4034-8b19-d9da093cda37 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:33.626929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4514d40-1a22-4034-8b19-d9da093cda37 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:33.627210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4514d40-1a22-4034-8b19-d9da093cda37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:33.627979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4514d40-1a22-4034-8b19-d9da093cda37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:33.631334 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4514d40-1a22-4034-8b19-d9da093cda37 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:33.653875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 489/1002] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:34.089304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355d02-e7f9-4225-b01b-a9435a52a9ec None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:34.089878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355d02-e7f9-4225-b01b-a9435a52a9ec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:34.089878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355d02-e7f9-4225-b01b-a9435a52a9ec None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:34.091418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 514/1003] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:34.096306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d088c-abb8-4000-a592-3eb196547cae None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:34.096602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d088c-abb8-4000-a592-3eb196547cae None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:34.096888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d088c-abb8-4000-a592-3eb196547cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:34.114144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b79d088c-abb8-4000-a592-3eb196547cae None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:34.115341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b79d088c-abb8-4000-a592-3eb196547cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:34.129084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:34.192171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 490/1004] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:34.214284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb355d02-e7f9-4225-b01b-a9435a52a9ec None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:34.284687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:34.321322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb355d02-e7f9-4225-b01b-a9435a52a9ec None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:34.358510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:34.359233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:34.359521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:34.359801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] PATH_INFO: `/v3/services/image` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:34.360459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:34.361829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:34.373726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:34.374880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:34.375786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:34.377354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 515/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Fri Jun 17 11:26:34 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:34.380956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d088c-abb8-4000-a592-3eb196547cae None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:34.488236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d088c-abb8-4000-a592-3eb196547cae None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:34.524131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f08ccc-3ed8-406e-891b-10cc3b480c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:34.525365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08ccc-3ed8-406e-891b-10cc3b480c6c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:34.525649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08ccc-3ed8-406e-891b-10cc3b480c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:34.525947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08ccc-3ed8-406e-891b-10cc3b480c6c None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:34.526647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08ccc-3ed8-406e-891b-10cc3b480c6c None admin] RBAC: Adding query filter params (name=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:34.526956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08ccc-3ed8-406e-891b-10cc3b480c6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:34.528312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08ccc-3ed8-406e-891b-10cc3b480c6c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:34.533311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 491/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:34.539085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:34.646948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3039db-7b5e-47dd-af79-1f4427353e47 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:34.678007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:34.683844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164926b-129b-410f-b827-939a7908b443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:34.684511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164926b-129b-410f-b827-939a7908b443 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:34.684795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164926b-129b-410f-b827-939a7908b443 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:34.685070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164926b-129b-410f-b827-939a7908b443 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:34.685775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164926b-129b-410f-b827-939a7908b443 None admin] RBAC: Adding query filter params (type=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:34.686129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164926b-129b-410f-b827-939a7908b443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:34.687483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164926b-129b-410f-b827-939a7908b443 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:34.697873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 516/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:35.386356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ee5aa-a803-43a6-ac53-3f56ba206679 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:35.386873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ee5aa-a803-43a6-ac53-3f56ba206679 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:35.386979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ee5aa-a803-43a6-ac53-3f56ba206679 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:35.388515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 492/1008] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:35.393598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3615697-5e08-48d8-b4b2-e2c0ff354297 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:35.394019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3615697-5e08-48d8-b4b2-e2c0ff354297 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:35.394205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3615697-5e08-48d8-b4b2-e2c0ff354297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:35.411272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c3615697-5e08-48d8-b4b2-e2c0ff354297 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:35.412456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c3615697-5e08-48d8-b4b2-e2c0ff354297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:35.483895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 517/1009] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:35.506807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ee5aa-a803-43a6-ac53-3f56ba206679 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:35.517715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:35.609768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ee5aa-a803-43a6-ac53-3f56ba206679 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:35.649276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a0e60-ce81-4818-8a93-87b539c74b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:35.650560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0e60-ce81-4818-8a93-87b539c74b49 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:35.650846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0e60-ce81-4818-8a93-87b539c74b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:35.651121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a0e60-ce81-4818-8a93-87b539c74b49 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:35.651772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a0e60-ce81-4818-8a93-87b539c74b49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:35.653129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a0e60-ce81-4818-8a93-87b539c74b49 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d25a0e60-ce81-4818-8a93-87b539c74b49 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:35.665819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:35.666826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:35.672584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 493/1010] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:35.675295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3615697-5e08-48d8-b4b2-e2c0ff354297 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:35.782298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3615697-5e08-48d8-b4b2-e2c0ff354297 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:35.809164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:35.825539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:35.825539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:35.825539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:35.825539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:35.825982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:35.826053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c03d9e-13dd-4228-9e7c-ec393bf86fd7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:35.826127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:35.826197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c03d9e-13dd-4228-9e7c-ec393bf86fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:35.826197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c03d9e-13dd-4228-9e7c-ec393bf86fd7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:35.827114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:35.828256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 494/1011] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:35.832559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0153e01-a782-4e23-bea6-dd2a78f84d23 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:35.832773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 518/1012] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:35.832879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0153e01-a782-4e23-bea6-dd2a78f84d23 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:35.833108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0153e01-a782-4e23-bea6-dd2a78f84d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:35.839263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:35.852259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e0153e01-a782-4e23-bea6-dd2a78f84d23 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:35.854394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e0153e01-a782-4e23-bea6-dd2a78f84d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:35.934737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 495/1013] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:35.955657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f9a3f-e6c0-4436-b3a5-37d8b5cd10fe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:35.958773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0153e01-a782-4e23-bea6-dd2a78f84d23 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:35.960863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:35.990724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:35.991392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:35.991672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:35.991950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:35.992653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:35.992959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:35.994395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:35.999547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 519/1014] 10.222.0.25 () {56 vars in 1112 bytes} [Fri Jun 17 11:26:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:36.005287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:36.064585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0153e01-a782-4e23-bea6-dd2a78f84d23 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:36.064930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:36.100415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:36.101212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:36.101502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:36.101784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:36.103091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:36.104516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:36.106164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 496/1015] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:36.111814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:36.117804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed692e-a98f-4978-9850-8c223a547bb5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:36.155233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ea59b-0821-4cee-8014-a7737b5e2e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:36.155895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ea59b-0821-4cee-8014-a7737b5e2e33 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:36.156181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ea59b-0821-4cee-8014-a7737b5e2e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:36.156460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ea59b-0821-4cee-8014-a7737b5e2e33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:36.157199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ea59b-0821-4cee-8014-a7737b5e2e33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:36.158575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ea59b-0821-4cee-8014-a7737b5e2e33 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:36.186146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 520/1016] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:26:36.190817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:36.226258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99331937-4ba1-4017-bb50-8e995b6c36c4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:36.269783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cee15-ba17-4674-81b7-2823649ab5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:36.270492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cee15-ba17-4674-81b7-2823649ab5db None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:36.270775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cee15-ba17-4674-81b7-2823649ab5db None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:36.271049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cee15-ba17-4674-81b7-2823649ab5db None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:36.271766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cee15-ba17-4674-81b7-2823649ab5db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:36.273211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cee15-ba17-4674-81b7-2823649ab5db None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:36.301568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 497/1017] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:36 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:36.867383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54ce4-d008-4604-bfed-4253b5f90d90 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:36.867922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54ce4-d008-4604-bfed-4253b5f90d90 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:36.867922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f54ce4-d008-4604-bfed-4253b5f90d90 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:36.870134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 521/1018] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:36.874991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b05eb-e049-47ff-b347-824ffe646a9d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:36.875366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b05eb-e049-47ff-b347-824ffe646a9d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:36.875551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b05eb-e049-47ff-b347-824ffe646a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:36.892208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cf5b05eb-e049-47ff-b347-824ffe646a9d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:36.893319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cf5b05eb-e049-47ff-b347-824ffe646a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:36.961907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 498/1019] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:36 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:36.983513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f54ce4-d008-4604-bfed-4253b5f90d90 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:37.088043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f54ce4-d008-4604-bfed-4253b5f90d90 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:37.099578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:37.125524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d30eb-323c-4eaa-9712-9e33bdad3e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:37.126226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d30eb-323c-4eaa-9712-9e33bdad3e60 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:37.126508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d30eb-323c-4eaa-9712-9e33bdad3e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:37.126783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d30eb-323c-4eaa-9712-9e33bdad3e60 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:37.127508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d30eb-323c-4eaa-9712-9e33bdad3e60 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:37.128862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d30eb-323c-4eaa-9712-9e33bdad3e60 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:37.150382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 522/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:26:36 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:38.437255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76139bc5-a41d-424d-ae7c-d24b4ee5a02c None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.438053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76139bc5-a41d-424d-ae7c-d24b4ee5a02c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.438053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76139bc5-a41d-424d-ae7c-d24b4ee5a02c None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.439442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 499/1021] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:38.444683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f4622-da4b-46a8-bd57-61c85514efc8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.445100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f4622-da4b-46a8-bd57-61c85514efc8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.445306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f4622-da4b-46a8-bd57-61c85514efc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.463926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bf6f4622-da4b-46a8-bd57-61c85514efc8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:38.465113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bf6f4622-da4b-46a8-bd57-61c85514efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:38.476772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc888-3da9-4024-aa95-5fd01f0e53f6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.477057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc888-3da9-4024-aa95-5fd01f0e53f6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.478150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc888-3da9-4024-aa95-5fd01f0e53f6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.479851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 500/1022] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:38.484902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de516d-7aca-4d8f-b864-846cc5f8b210 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.485190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de516d-7aca-4d8f-b864-846cc5f8b210 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.485470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de516d-7aca-4d8f-b864-846cc5f8b210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.503170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-13de516d-7aca-4d8f-b864-846cc5f8b210 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:38.504354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-13de516d-7aca-4d8f-b864-846cc5f8b210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:38.548726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:38.557859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 523/1023] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:38.573685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f4622-da4b-46a8-bd57-61c85514efc8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:38.574265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 501/1024] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:38.601489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de516d-7aca-4d8f-b864-846cc5f8b210 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:38.676980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:38.685728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f4622-da4b-46a8-bd57-61c85514efc8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:38.713144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de516d-7aca-4d8f-b864-846cc5f8b210 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:38.721194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:38.722041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.722328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.722604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.734027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:38.742228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:38.743194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:38.744043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:38.748473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 524/1025] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Jun 17 11:26:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:38.749306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:38.754743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:38.755806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.756290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.756758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.758721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:38.760843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:38.764526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 502/1026] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:38.768823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:38.817288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:38.859924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f92c6-ce17-471b-ac49-f236614f14ae None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:38.873777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53baf8-cbee-4d7f-bebd-c6239635c357 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:38.901144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:38.901916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.902133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.902411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.903166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:38.903478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:38.904846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:38.911331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:38.912006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:38.912289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:38.912563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:38.913272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:38.918248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 525/1027] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Jun 17 11:26:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:38.918377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:38.919153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:38.945867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 503/1028] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:26:38 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:26:38.951885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:39.044174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1224c-4746-42ac-8289-fa14ba4869da None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.080640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07ac7c-f14b-45ec-9b26-e88e25773c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.081453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ac7c-f14b-45ec-9b26-e88e25773c5f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.081737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ac7c-f14b-45ec-9b26-e88e25773c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.082042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07ac7c-f14b-45ec-9b26-e88e25773c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.094397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07ac7c-f14b-45ec-9b26-e88e25773c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.095862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07ac7c-f14b-45ec-9b26-e88e25773c5f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-dd07ac7c-f14b-45ec-9b26-e88e25773c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:39.115245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:39.116226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:39.117131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:39.117131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:39.117131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:39.117131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:39.117131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 526/1029] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:39.123853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.236644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ebb85-34c4-4681-bf86-b5e6b023171e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.238245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:39.262519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b22c-c4bf-4a43-baa0-99cf4ea548c7 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.262813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b22c-c4bf-4a43-baa0-99cf4ea548c7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.263088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b22c-c4bf-4a43-baa0-99cf4ea548c7 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.264613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 527/1030] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:39.269443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4e3b9-23e7-449a-b081-f59e9c1189d5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.269740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4e3b9-23e7-449a-b081-f59e9c1189d5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.270365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4e3b9-23e7-449a-b081-f59e9c1189d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.275067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.275739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.276031 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.276286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.277010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:39.277312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.278720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.288697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c4e4e3b9-23e7-449a-b081-f59e9c1189d5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:39.289953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c4e4e3b9-23e7-449a-b081-f59e9c1189d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:39.295868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 504/1031] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:39.296766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.362710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:39.371281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 528/1032] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:39.394751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4e3b9-23e7-449a-b081-f59e9c1189d5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.416604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e386-7502-4928-98f8-207f25caf079 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.455695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.457171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.457455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.457740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.465407 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.466874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:39.482581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:39.483582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:39.484525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 505/1033] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:39.489671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.502948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4e3b9-23e7-449a-b081-f59e9c1189d5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.517512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:39.540095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.540894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.541181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.541454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.542132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.543538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:39.555151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:39.556780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:39.557628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 529/1034] 10.222.0.25 () {56 vars in 1105 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:39.562562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.604847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce61fa-555d-4608-9f6b-6b80c13cada5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.630883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:39.645650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.646481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.646685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.646943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.647824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:39.648141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.649605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.668256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 506/1035] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:39.673595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.674822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782889c-edd5-4cd5-bf07-d1ae4a4204f2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.713166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.713853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.714149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.714430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.715140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] RBAC: Adding query filter params (name=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:39.715458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.716833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.722132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 530/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:39.727998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.759637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:39.779871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc26f6-5950-453a-be2c-3a0bfe987541 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.817776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.818370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.818657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.818928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.834083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c4f0c-18b3-4428-8da2-862228894f5a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:39.870061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:39.880222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:39.880480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:39.880763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:39.881035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:39.881735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] RBAC: Adding query filter params (type=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:39.882072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.883421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.888845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 531/1037] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:39.894692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:39.906171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:39.907953 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:39.995020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:40.010700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d074c92-abe0-4524-b749-3b70b19f677c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:40.015920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 507/1038] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:26:40.021494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:40.051605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:40.052264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:40.052549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:40.052825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:40.053615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] RBAC: Adding query filter params (interface=public, service_id=cefa044a466348fa87713063dd85f6b1, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:40.053969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:40.055317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:40.065645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 532/1039] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:26:39 2022] GET /identity/v3/endpoints?service_id=cefa044a466348fa87713063dd85f6b1&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:40.067620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:40.137138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:40.139725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d7718-a2c9-4c01-9577-5cf30d5fbd6c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:40.178237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:40.179155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:40.179339 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:40.179618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:40.191348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bcca0-b459-479a-bece-6aea675d77c9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:40.191553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:40.193029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:40.212936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:40.213934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:40.214849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:40.214849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:40.214849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:40.214849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:40.214849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 508/1040] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:40.219645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:40.229911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:40.230593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:40.230880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:40.231159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:40.231849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:40.233221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:40.240245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 533/1041] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:26:40 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:26:40.252511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:40.329171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bc506-c02d-4c5c-a175-ede02e51c2b1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:40.366127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:40.366831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:40.367114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:40.367388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:40.368147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:40.368453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:40.369827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:40.385907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 509/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:40.388322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:40.493199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b35fbf-7b8b-48a9-bdd4-d098ae4d82b7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:40.514523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:40.530645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:40.531444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:40.531723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:40.531995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:40.539044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:40.540498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:40.554455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:40.555318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:40.556200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 534/1043] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:40.561680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:40.662341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:40.671936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b709aff-d43a-4a1a-be0a-6cbc91e89c90 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:40.705184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf8f01-bcb6-4300-94ec-fb1da872d2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:40.705873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf8f01-bcb6-4300-94ec-fb1da872d2db None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:40.706159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf8f01-bcb6-4300-94ec-fb1da872d2db None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:40.706431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf8f01-bcb6-4300-94ec-fb1da872d2db None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:40.707214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf8f01-bcb6-4300-94ec-fb1da872d2db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:40.707520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf8f01-bcb6-4300-94ec-fb1da872d2db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:40.708960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf8f01-bcb6-4300-94ec-fb1da872d2db None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:40.722177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 510/1044] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:40.728375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:40.832765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ec30-7ebc-4c45-8b4a-2afef7ed386a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:40.866015 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694eaf-d12f-4824-9e01-f8f49ac37cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:40.866729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694eaf-d12f-4824-9e01-f8f49ac37cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:40.867017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694eaf-d12f-4824-9e01-f8f49ac37cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:40.867292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694eaf-d12f-4824-9e01-f8f49ac37cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:40.868754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694eaf-d12f-4824-9e01-f8f49ac37cf8 None admin] RBAC: Adding query filter params (role.id=d4c16bd9276449d887ea030f63a2cde1, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=804a35678c91438eae36f61c1de8c579) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:40.869074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694eaf-d12f-4824-9e01-f8f49ac37cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:40.870518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694eaf-d12f-4824-9e01-f8f49ac37cf8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:40.882110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 535/1045] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:40 2022] GET /identity/v3/role_assignments?user.id=804a35678c91438eae36f61c1de8c579&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=d4c16bd9276449d887ea030f63a2cde1 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:41.159879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea457d20-8ea8-4e49-88b2-80fd59eba621 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:41.160252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea457d20-8ea8-4e49-88b2-80fd59eba621 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:41.160449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea457d20-8ea8-4e49-88b2-80fd59eba621 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:41.162175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 511/1046] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:41.167934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9033-3fda-4c4a-98af-e7cf2a5c70a1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:41.168227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9033-3fda-4c4a-98af-e7cf2a5c70a1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:41.168506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9033-3fda-4c4a-98af-e7cf2a5c70a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:41.185988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-741c9033-3fda-4c4a-98af-e7cf2a5c70a1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:41.187179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-741c9033-3fda-4c4a-98af-e7cf2a5c70a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:41.226764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:41.255944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 536/1047] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:41.279501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea457d20-8ea8-4e49-88b2-80fd59eba621 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:41.389385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea457d20-8ea8-4e49-88b2-80fd59eba621 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:41.425876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:41.426594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:41.426880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:41.427158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:41.438244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:41.445811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:41.446753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:41.452850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 512/1048] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:41.455263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9033-3fda-4c4a-98af-e7cf2a5c70a1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:41.560460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9033-3fda-4c4a-98af-e7cf2a5c70a1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:41.594582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:41.595252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:41.595589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:41.595862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:41.596647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:41.596959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:41.598348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:41.611530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 537/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:41.612668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:41.629157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:41.720740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824a6dd-5c5f-4a19-9750-ea76c695edff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:41.755543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:41.756668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:41.757122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:41.757563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] PATH_INFO: `/v3/users/d1b2f2e75bae49cebd525d6d9fcc852e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:41.771502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] RBAC: Authorizing `identity:get_user(user_id=d1b2f2e75bae49cebd525d6d9fcc852e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:41.773127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:41.775885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 513/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:41 2022] GET /identity/v3/users/d1b2f2e75bae49cebd525d6d9fcc852e => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:41.781467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:41.819005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:41.886438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91e534-37af-4252-82ae-66f70ede3c64 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:41.920698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:41.921352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:41.921639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:41.921944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:41.994307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:41.995802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:42.030820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:42.102938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 538/1051] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:42.104130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:42.174214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:42.206254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55a037-969c-4c0f-b9e9-d869fb888e07 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:42.239294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4833e5-04d8-4e7a-abf0-590bc2ec59df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:42.240148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4833e5-04d8-4e7a-abf0-590bc2ec59df None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:42.240429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4833e5-04d8-4e7a-abf0-590bc2ec59df None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:42.240702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4833e5-04d8-4e7a-abf0-590bc2ec59df None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:42.241891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4833e5-04d8-4e7a-abf0-590bc2ec59df None admin] RBAC: Authorizing `identity:get_project(project_id=8fa8c2fb977e410ea9e74cffb11e6d88)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:42.243342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4833e5-04d8-4e7a-abf0-590bc2ec59df None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:42.245236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 514/1052] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:26:42 2022] GET /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:42.251117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:42.353878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed879e8c-fcf0-44c4-a2d7-43146b4815e4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:42.375505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385683e-fea0-4fd2-9050-bdcdc5003409 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:42.375796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385683e-fea0-4fd2-9050-bdcdc5003409 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:42.376078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385683e-fea0-4fd2-9050-bdcdc5003409 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:42.377604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 515/1053] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:42.382314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4099d7c-6cbf-4f87-8b5e-4a0c870ce949 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:42.382604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4099d7c-6cbf-4f87-8b5e-4a0c870ce949 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:42.382882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4099d7c-6cbf-4f87-8b5e-4a0c870ce949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:42.390682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:42.391337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:42.391617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:42.392166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:42.393001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88, user.id=d1b2f2e75bae49cebd525d6d9fcc852e) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:42.393311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:42.394754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:42.397658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:42.400048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e4099d7c-6cbf-4f87-8b5e-4a0c870ce949 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:42.401345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e4099d7c-6cbf-4f87-8b5e-4a0c870ce949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:42.406040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 539/1054] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:42 2022] GET /identity/v3/role_assignments?user.id=d1b2f2e75bae49cebd525d6d9fcc852e&scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:42.475317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 516/1055] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:42.503295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:42.617028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93351e36-86b4-4dcd-8a81-5f9b071b82cc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:42.656753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:42.657551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:42.657851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:42.658134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] PATH_INFO: `/v3/services/image` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:42.658817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:42.660177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:42.672738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:42.673743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:42.675378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 540/1056] 10.222.0.25 () {56 vars in 1105 bytes} [Fri Jun 17 11:26:42 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:42.679788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4099d7c-6cbf-4f87-8b5e-4a0c870ce949 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:42.749856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:42.785859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4099d7c-6cbf-4f87-8b5e-4a0c870ce949 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:42.821047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:42.821723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:42.822034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:42.822305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:42.823011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:42.823316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:42.824675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:42.834369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 517/1057] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:42.835570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:42.902143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:42.943164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be810bb-53b1-4c89-95c6-3af49601ac45 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:42.980053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2154f6-442c-471e-a643-e19537deb00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:42.980745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2154f6-442c-471e-a643-e19537deb00c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:42.981031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2154f6-442c-471e-a643-e19537deb00c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:42.981307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2154f6-442c-471e-a643-e19537deb00c None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:42.982083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2154f6-442c-471e-a643-e19537deb00c None admin] RBAC: Adding query filter params (type=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:42.982396 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2154f6-442c-471e-a643-e19537deb00c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:42.983896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2154f6-442c-471e-a643-e19537deb00c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:42.994521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 541/1058] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:42.995989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:43.022509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d290ca4-49bf-40a7-8df3-4b9983f64aec None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:43.022928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d290ca4-49bf-40a7-8df3-4b9983f64aec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:43.023346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d290ca4-49bf-40a7-8df3-4b9983f64aec None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:43.025871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 542/1059] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:43.031116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a65c8e-49fa-410c-9539-4bd5cf9383b4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:43.031577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a65c8e-49fa-410c-9539-4bd5cf9383b4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:43.031866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a65c8e-49fa-410c-9539-4bd5cf9383b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:43.050738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-44a65c8e-49fa-410c-9539-4bd5cf9383b4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:43.051848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-44a65c8e-49fa-410c-9539-4bd5cf9383b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:43.106629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad380cc9-6c53-401a-9d3f-c484db09b9b7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:43.133038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 543/1060] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:43.139599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:43.146151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:43.147237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:43.147524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:43.147804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:43.148566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:43.150512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:43.161373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a65c8e-49fa-410c-9539-4bd5cf9383b4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:43.174401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 518/1061] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:26:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:26:43.235381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:43.284668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a65c8e-49fa-410c-9539-4bd5cf9383b4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:43.331057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:43.332164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:43.332596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:43.333026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:43.346787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:43.356560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:43.357681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:43.358783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:43.360940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 544/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Jun 17 11:26:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:43.367517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:43.481687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca86ff4-c1ae-44ac-957a-c39e613602c3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:43.517141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:43.518212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:43.518680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:43.519154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:43.520130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:43.520647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:43.522259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:43.532583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 519/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Jun 17 11:26:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:43.538397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:43.655060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acb46a-6494-4c28-8a79-1563d2fa77f0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:43.670163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:43.692032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:43.692841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:43.693125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:43.693398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:43.705297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:43.706773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:43.726631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:43.728134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:43.729061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:43.729061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:43.729061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:43.729061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:43.729061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 545/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:43.735779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:43.794827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:43.851791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ac1a8-7f96-488b-8634-614d9df3ef9f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:43.893658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:43.894601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:43.895036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:43.895458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:43.896341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:43.896795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:43.898354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:43.912220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 520/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:43.918045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.037784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ebe90-d44c-4c71-b458-26d1f895f3ed None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:44.071383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:44.072172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.072449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.072717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.080147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:44.081569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:44.095749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:44.096665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:44.097613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:44.098778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 546/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:26:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:44.103271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.212451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a12ec1-955a-461c-a637-8f9bca87dc7a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:44.223614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:44.251662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:44.252345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.252632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.252911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.253757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:44.254098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:44.255523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:44.278739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 521/1067] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:26:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:44.279922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.310094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:44.387288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f6c62-220a-4d2d-aa07-60758e5b1c87 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:44.421569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:44.422399 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.422761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.423042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.434544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:44.436014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:44.453001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:44.454061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:44.459945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:44.459945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:44.459945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:44.459945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:44.459945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 547/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:44.460242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.561348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4646c8d-044f-4626-8070-fd7730619c58 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:44.598521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81197626-9d5b-4b38-8563-80fa43547cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:44.599214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197626-9d5b-4b38-8563-80fa43547cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.599500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197626-9d5b-4b38-8563-80fa43547cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.599777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81197626-9d5b-4b38-8563-80fa43547cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.600519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197626-9d5b-4b38-8563-80fa43547cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:44.600830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197626-9d5b-4b38-8563-80fa43547cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:44.602856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81197626-9d5b-4b38-8563-80fa43547cc4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:44.619503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 522/1069] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:44.622940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.643924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b89cc-bb49-4ec8-9236-7e1968474835 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.644226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b89cc-bb49-4ec8-9236-7e1968474835 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.644791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b89cc-bb49-4ec8-9236-7e1968474835 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.646568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 523/1070] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:44.651780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35774-c97e-4e68-8cc0-c0e1db7a3618 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.652072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35774-c97e-4e68-8cc0-c0e1db7a3618 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.652356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35774-c97e-4e68-8cc0-c0e1db7a3618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.669478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-01d35774-c97e-4e68-8cc0-c0e1db7a3618 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:44.670702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-01d35774-c97e-4e68-8cc0-c0e1db7a3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:44.694978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:44.715595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:44.741596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76ab93-5046-4ac4-87ef-5a0e677ed4cc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:44.748641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 524/1071] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:44.770455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d35774-c97e-4e68-8cc0-c0e1db7a3618 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.777959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:44.778765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.779464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.779743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.788260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:44.790262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:44.804485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:44.805408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:44.806346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:44.810604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 548/1072] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:44.811269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.885146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d35774-c97e-4e68-8cc0-c0e1db7a3618 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:44.913385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6e4cb-af06-4d7d-8523-5ef81dd92428 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:44.924639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:44.925445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.925727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.926030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.937947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:44.945564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:44.947272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:44.948538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 525/1073] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:44.950929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:44.951600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:44.951886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:44.952411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:44.953018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:44.953250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.953419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:44.954773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:44.963037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:44.968725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 549/1074] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:44.974111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:44.976294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:45.057492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f228b-a623-4595-86eb-6acebc101732 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:45.084987 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2487d69-6865-40d8-9d32-b0c20942c91f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:45.091408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:45.092083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.092371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.092664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.093401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:45.093708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:45.095109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:45.104133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 526/1075] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:45.109793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:45.122259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:45.123154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.123437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.123712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/804a35678c91438eae36f61c1de8c579/roles/d4c16bd9276449d887ea030f63a2cde1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.126071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=804a35678c91438eae36f61c1de8c579, role_id=d4c16bd9276449d887ea030f63a2cde1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:45.127496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:45.141973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 550/1076] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:26:44 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/804a35678c91438eae36f61c1de8c579/roles/d4c16bd9276449d887ea030f63a2cde1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:26:45.203971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:45.226280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29c0cb-d928-4271-95ce-8224aea22f9a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:45.264254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9555f2b-3cdf-44fe-a9a3-a52b296ef5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:45.265412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9555f2b-3cdf-44fe-a9a3-a52b296ef5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.265896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9555f2b-3cdf-44fe-a9a3-a52b296ef5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.266324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9555f2b-3cdf-44fe-a9a3-a52b296ef5c5 None admin] PATH_INFO: `/v3/users/d1b2f2e75bae49cebd525d6d9fcc852e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.267774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9555f2b-3cdf-44fe-a9a3-a52b296ef5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=d1b2f2e75bae49cebd525d6d9fcc852e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:45.269479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9555f2b-3cdf-44fe-a9a3-a52b296ef5c5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:45.271765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 527/1077] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:45 2022] GET /identity/v3/users/d1b2f2e75bae49cebd525d6d9fcc852e => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:45.278106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:45.346323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:45.393143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b5c50-4354-49bd-af3d-eb8502c64a70 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:45.396448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f590d-4ed4-47bb-819c-12bf12f87114 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.396741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f590d-4ed4-47bb-819c-12bf12f87114 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.397023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f590d-4ed4-47bb-819c-12bf12f87114 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.398571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 528/1078] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:45.403643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8068c3-0c50-4538-9f61-4effcb11fbea None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.403937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8068c3-0c50-4538-9f61-4effcb11fbea None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.404224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8068c3-0c50-4538-9f61-4effcb11fbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.421626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0b8068c3-0c50-4538-9f61-4effcb11fbea None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:45.422879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0b8068c3-0c50-4538-9f61-4effcb11fbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:45.430875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:45.431753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.432112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.432298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.433514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa8c2fb977e410ea9e74cffb11e6d88)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:45.434983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:45.436867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 551/1079] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:26:45 2022] GET /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:45.442602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:45.499314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 529/1080] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:45.521377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8068c3-0c50-4538-9f61-4effcb11fbea None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:45.552849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc318f-0f1e-4179-9c27-15d155261f26 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:45.580034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:45.590198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:45.591099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.591378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.591658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88/users/d1b2f2e75bae49cebd525d6d9fcc852e/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.593670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fa8c2fb977e410ea9e74cffb11e6d88, user_id=d1b2f2e75bae49cebd525d6d9fcc852e, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:45.595139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:45.606795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:45.617868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 552/1081] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:26:45 2022] PUT /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88/users/d1b2f2e75bae49cebd525d6d9fcc852e/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:26:45.645887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8068c3-0c50-4538-9f61-4effcb11fbea None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:45.685023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:45.685702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:45.686036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:45.686319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:45.759740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:45.761229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:45.846247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:45.869632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 530/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:45.875206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:45.978661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6e1eb-5680-41ea-8f78-3ed4acc18825 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:46.012010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d8498-8adb-411e-b3b3-b887d0876688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:46.012805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8498-8adb-411e-b3b3-b887d0876688 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:46.013086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8498-8adb-411e-b3b3-b887d0876688 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:46.013355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8498-8adb-411e-b3b3-b887d0876688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:46.024694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d8498-8adb-411e-b3b3-b887d0876688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:46.026156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d8498-8adb-411e-b3b3-b887d0876688 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-718d8498-8adb-411e-b3b3-b887d0876688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:46.046197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:46.047138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:46.048052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:46.048052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:46.048052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:46.048052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:46.048052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 553/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:46.054297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:46.159298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c6da86-acc7-4868-b9f1-634e94f297fb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:46.193963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fef075-e7dd-44fe-acb2-ab5903b5d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:46.194636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fef075-e7dd-44fe-acb2-ab5903b5d13d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:46.194924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fef075-e7dd-44fe-acb2-ab5903b5d13d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:46.195200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fef075-e7dd-44fe-acb2-ab5903b5d13d None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:46.196472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fef075-e7dd-44fe-acb2-ab5903b5d13d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:46.196781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fef075-e7dd-44fe-acb2-ab5903b5d13d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:46.198164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fef075-e7dd-44fe-acb2-ab5903b5d13d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:46.209577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 531/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:47.293663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0bbfb-fb73-4e81-b43a-a6c4e191510d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:47.294227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0bbfb-fb73-4e81-b43a-a6c4e191510d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:47.294227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0bbfb-fb73-4e81-b43a-a6c4e191510d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:47.295805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 554/1085] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:47.300898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41efe1-bdbf-4f56-80b3-2081c455b2d5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:47.301335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41efe1-bdbf-4f56-80b3-2081c455b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:47.301595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41efe1-bdbf-4f56-80b3-2081c455b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:47.318303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8c41efe1-bdbf-4f56-80b3-2081c455b2d5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:47.319491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c41efe1-bdbf-4f56-80b3-2081c455b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:47.349432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:47.387976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 532/1086] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:47.411879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0bbfb-fb73-4e81-b43a-a6c4e191510d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:47.453160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:47.514452 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0bbfb-fb73-4e81-b43a-a6c4e191510d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:47.548006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:47.548750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:47.549034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:47.549307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:47.560435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:47.568502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:47.569609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 17 11:26:47.570519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:47.573980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 555/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Fri Jun 17 11:26:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:47.576087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41efe1-bdbf-4f56-80b3-2081c455b2d5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:47.675619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41efe1-bdbf-4f56-80b3-2081c455b2d5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:47.709044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75c9f0-0073-4250-bf32-25d115aed575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:47.709747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75c9f0-0073-4250-bf32-25d115aed575 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:47.710064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75c9f0-0073-4250-bf32-25d115aed575 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:47.710345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75c9f0-0073-4250-bf32-25d115aed575 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:47.711117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75c9f0-0073-4250-bf32-25d115aed575 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:47.711469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75c9f0-0073-4250-bf32-25d115aed575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:47.712895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75c9f0-0073-4250-bf32-25d115aed575 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:47.722310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 533/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Jun 17 11:26:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:47.728431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:47.784227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65192840-c1ff-4eef-8e7a-f3f0f3e9fae6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:47.784527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65192840-c1ff-4eef-8e7a-f3f0f3e9fae6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:47.784808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65192840-c1ff-4eef-8e7a-f3f0f3e9fae6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:47.786495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 534/1089] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:26:47.791414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6599f-e2b9-426e-bcd6-6bb6ecf156b1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:47.791705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6599f-e2b9-426e-bcd6-6bb6ecf156b1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:47.791985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6599f-e2b9-426e-bcd6-6bb6ecf156b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:47.808614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f2a6599f-e2b9-426e-bcd6-6bb6ecf156b1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:47.809740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f2a6599f-e2b9-426e-bcd6-6bb6ecf156b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:47.826544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:47.830488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344bfcf-2b11-46e1-ab1e-8bc528c7f66d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:47.833532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:47.865012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:47.865798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:47.866139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:47.866441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:47.876748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:47.878202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:47.878488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 535/1090] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:47.895827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:47.896728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:47.897631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:47.897631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:47.897631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:47.897631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:47.897631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 556/1091] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:47.902535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:47.902668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6599f-e2b9-426e-bcd6-6bb6ecf156b1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:47.998900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6599f-e2b9-426e-bcd6-6bb6ecf156b1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:47.999522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a0f33-3c79-419b-840b-52a1d1517618 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.031296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.032093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.032380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.032655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.032746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.033336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.033624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.033932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.034662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:48.034971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.036597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.044940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:26:48.048743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 557/1092] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:26:48.053613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:48.054818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:26:48.056298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:48.056938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.057012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 536/1093] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:26:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:48.061330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.067833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.083947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.153896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561eb78-5b24-4ef2-bc9f-d5e928db2281 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.162582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a61982-873a-455d-9605-5ea0918a7602 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.191379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.192177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.192459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.192735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.196831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.197641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.197956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.198234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.198988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:48.199295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.200665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.200778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.202109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.208996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 537/1094] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:48.214858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:26:48.215881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:48.216915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:48.218373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 558/1095] 10.222.0.25 () {56 vars in 1093 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:48.222648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.292786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.298430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.316647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4def-b81b-47a8-b4cd-8fd1ecbc8b94 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.324413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c285f-19dc-4a04-9f13-464c1d0efae8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.350354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.351252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.351579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.351858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] PATH_INFO: `/v3/users/d1b2f2e75bae49cebd525d6d9fcc852e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.353654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] RBAC: Authorizing `identity:get_user(user_id=d1b2f2e75bae49cebd525d6d9fcc852e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.355107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.356728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 538/1096] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/users/d1b2f2e75bae49cebd525d6d9fcc852e => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:48.362310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.363835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.364517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.364801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.365074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.365953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:48.366268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.367708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.385806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 559/1097] 10.222.0.25 () {56 vars in 1133 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:48.391306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.466422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3486e0-aff9-4de7-8d6c-29f26c58cc13 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.490229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea1671-ff97-4cdd-a01c-b7c3f0c9a58f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.499222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.499898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.500180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.500453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.515786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.518968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.524713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.525360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.525637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.525952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.568067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.569532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.593618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.595081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.671481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 539/1098] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:26:48.677166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.695332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 560/1099] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 17 11:26:48.701237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.725079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.727747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.776808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42621f-b681-42fa-a752-8deb36c5bac1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.798295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b68f6-4232-4512-b017-1eb967165a6a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.809696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.810583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.810861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.811131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.812289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] RBAC: Authorizing `identity:get_project(project_id=8fa8c2fb977e410ea9e74cffb11e6d88)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.813709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.815593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 540/1100] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:48.821889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.833193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.834030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.834314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.834594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.845231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.846693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:26:48.864087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:48.865029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:48.866365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:26:48.866365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:26:48.866365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:26:48.866365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:26:48.866365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 561/1101] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:26:48.870699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:48.920086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06126260-3b06-4a1e-bd3c-0e8c2272514c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:48.936903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.950046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:48.954616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:48.955268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:48.955539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:48.955816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:48.956640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88, user.id=d1b2f2e75bae49cebd525d6d9fcc852e) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:48.956956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:48.958412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:48.964758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 541/1102] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/role_assignments?user.id=d1b2f2e75bae49cebd525d6d9fcc852e&scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:48.971352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb4392-3525-4ac4-8496-0033782f3b84 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:49.008902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:49.009583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:49.009891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:49.010170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:49.010912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:49.011225 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:49.012643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:49.026532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 562/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:26:49.032270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:49.135924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dfee1-7a3e-4352-8ff3-8368d7798404 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:49.169242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:49.170115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:49.170404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:49.170684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:49.177432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:49.178933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:26:49.193361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:26:49.194310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:26:49.195197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 542/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:26:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:26:49.200873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:49.298593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ea41-c529-4944-ae33-3729f005a70c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:49.331743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e8669-1a5b-4869-bbc3-60c35526c1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:49.332401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e8669-1a5b-4869-bbc3-60c35526c1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:49.332683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e8669-1a5b-4869-bbc3-60c35526c1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:49.332957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e8669-1a5b-4869-bbc3-60c35526c1a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:49.333744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e8669-1a5b-4869-bbc3-60c35526c1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:49.334079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e8669-1a5b-4869-bbc3-60c35526c1a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:49.335486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e8669-1a5b-4869-bbc3-60c35526c1a0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:49.344441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:49.347750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 563/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:26:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:49.353801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:49.397363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:49.452808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f876da-e6b5-4c23-8ba6-3ae44f9db0a7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:49.486516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c630c-9fad-4765-8bdf-7118f1099170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:49.487177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c630c-9fad-4765-8bdf-7118f1099170 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:49.487462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c630c-9fad-4765-8bdf-7118f1099170 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:49.487738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c630c-9fad-4765-8bdf-7118f1099170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:49.488596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c630c-9fad-4765-8bdf-7118f1099170 None admin] RBAC: Adding query filter params (role.id=d4c16bd9276449d887ea030f63a2cde1, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=804a35678c91438eae36f61c1de8c579) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:49.488911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c630c-9fad-4765-8bdf-7118f1099170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:49.490381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c630c-9fad-4765-8bdf-7118f1099170 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:49.496827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 543/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:49 2022] GET /identity/v3/role_assignments?user.id=804a35678c91438eae36f61c1de8c579&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=d4c16bd9276449d887ea030f63a2cde1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:51.122055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43346eb-38de-4979-8091-5bb42fe45ffb None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:51.122572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43346eb-38de-4979-8091-5bb42fe45ffb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:51.122665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43346eb-38de-4979-8091-5bb42fe45ffb None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:51.123986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 564/1107] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:51.129102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bdb21-5c8f-4b79-b091-ed2f5e8d59ba None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:51.129504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bdb21-5c8f-4b79-b091-ed2f5e8d59ba None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:51.129662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bdb21-5c8f-4b79-b091-ed2f5e8d59ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:51.146596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ab9bdb21-5c8f-4b79-b091-ed2f5e8d59ba None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:51.147774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ab9bdb21-5c8f-4b79-b091-ed2f5e8d59ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:51.213023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 544/1108] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:51.237214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43346eb-38de-4979-8091-5bb42fe45ffb None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:51.340342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43346eb-38de-4979-8091-5bb42fe45ffb None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:51.374013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5e866-08be-4a4e-8dfe-b80c4385b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:51.374740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5e866-08be-4a4e-8dfe-b80c4385b6cc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:51.375029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5e866-08be-4a4e-8dfe-b80c4385b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:51.375305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5e866-08be-4a4e-8dfe-b80c4385b6cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:51.376465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5e866-08be-4a4e-8dfe-b80c4385b6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:51.377895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5e866-08be-4a4e-8dfe-b80c4385b6cc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:51.379492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 565/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:26:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:51.385968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bdb21-5c8f-4b79-b091-ed2f5e8d59ba None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:51.419075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:51.481536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bdb21-5c8f-4b79-b091-ed2f5e8d59ba None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:51.515161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35791b05-803c-4e7d-ab7a-eea8916f1920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:51.515829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35791b05-803c-4e7d-ab7a-eea8916f1920 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:51.516114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35791b05-803c-4e7d-ab7a-eea8916f1920 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:51.516393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35791b05-803c-4e7d-ab7a-eea8916f1920 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:51.517102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35791b05-803c-4e7d-ab7a-eea8916f1920 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:51.518568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35791b05-803c-4e7d-ab7a-eea8916f1920 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:51.537182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 545/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:26:51 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:26:53.674670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9274-e2e5-4b73-83c8-111f83cb13ff None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:53.675174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9274-e2e5-4b73-83c8-111f83cb13ff None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:53.675284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9274-e2e5-4b73-83c8-111f83cb13ff None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:53.676602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 566/1111] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:53.681950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e3adb-ac58-4771-b07b-57fcc7920e15 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:53.682335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e3adb-ac58-4771-b07b-57fcc7920e15 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:53.682498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e3adb-ac58-4771-b07b-57fcc7920e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:53.698635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-449e3adb-ac58-4771-b07b-57fcc7920e15 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:53.699801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-449e3adb-ac58-4771-b07b-57fcc7920e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:53.763119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 546/1112] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:53.786859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a9274-e2e5-4b73-83c8-111f83cb13ff None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:53.854322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:53.888247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a9274-e2e5-4b73-83c8-111f83cb13ff None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:53.921649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:53.922553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:53.922840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:53.923116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] PATH_INFO: `/v3/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:53.924588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] RBAC: Authorizing `identity:get_role(role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:53.925981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:53.927355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 567/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:53 2022] GET /identity/v3/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:53.933092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e3adb-ac58-4771-b07b-57fcc7920e15 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:53.956544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:54.029940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e3adb-ac58-4771-b07b-57fcc7920e15 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:54.062520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:54.063379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:54.063660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:54.063934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] PATH_INFO: `/v3/users/1282abf1a2b2402889d1b148890e5b69` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:54.076928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] RBAC: Authorizing `identity:get_user(user_id=1282abf1a2b2402889d1b148890e5b69)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:54.078395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:54.080012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 547/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:53 2022] GET /identity/v3/users/1282abf1a2b2402889d1b148890e5b69 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:54.085647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:54.187482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85a6b1-c22c-4229-a716-65d5efe8f457 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:54.220416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:54.221052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:54.221332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:54.221604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:54.224919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:54.287839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:54.289295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:54.385552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 568/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:26:54.391452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:54.469664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:54.488616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08286719-de32-4ae4-abda-2460728b9a2c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:54.522057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b376d1-5057-4747-9639-7cad36be77e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:54.522901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b376d1-5057-4747-9639-7cad36be77e4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:54.523187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b376d1-5057-4747-9639-7cad36be77e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:54.523492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b376d1-5057-4747-9639-7cad36be77e4 None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:54.524669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b376d1-5057-4747-9639-7cad36be77e4 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa8c2fb977e410ea9e74cffb11e6d88)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:54.526126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b376d1-5057-4747-9639-7cad36be77e4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:54.527812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 548/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:26:54 2022] GET /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:54.533682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:54.568431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:54.633738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d87aaa-83c4-43c8-8016-52c0c6cb0b51 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:54.666993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b213aca-c383-4cd4-9a5f-4f131c2181d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:54.667641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b213aca-c383-4cd4-9a5f-4f131c2181d7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:54.667926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b213aca-c383-4cd4-9a5f-4f131c2181d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:54.668203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b213aca-c383-4cd4-9a5f-4f131c2181d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:54.669032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b213aca-c383-4cd4-9a5f-4f131c2181d7 None admin] RBAC: Adding query filter params (role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88, user.id=1282abf1a2b2402889d1b148890e5b69) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:26:54.669346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b213aca-c383-4cd4-9a5f-4f131c2181d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:54.670780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b213aca-c383-4cd4-9a5f-4f131c2181d7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:54.676091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 569/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:26:54 2022] GET /identity/v3/role_assignments?user.id=1282abf1a2b2402889d1b148890e5b69&scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:56.776641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfa9c5-4ee5-4df5-b395-ed58d3632244 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:56.777153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfa9c5-4ee5-4df5-b395-ed58d3632244 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:56.777265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfa9c5-4ee5-4df5-b395-ed58d3632244 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:56.778634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 549/1118] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:56.783820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8dc40-b43a-4555-a6a6-d9a3819a12b0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:56.784211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8dc40-b43a-4555-a6a6-d9a3819a12b0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:56.784370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8dc40-b43a-4555-a6a6-d9a3819a12b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:56.801429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b1e8dc40-b43a-4555-a6a6-d9a3819a12b0 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:56.802626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b1e8dc40-b43a-4555-a6a6-d9a3819a12b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:56.866290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 570/1119] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:56.888620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfa9c5-4ee5-4df5-b395-ed58d3632244 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:56.987299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfa9c5-4ee5-4df5-b395-ed58d3632244 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:57.019678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:57.020520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:57.020804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:57.021078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] PATH_INFO: `/v3/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:57.022567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] RBAC: Authorizing `identity:get_role(role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:57.024528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:57.026091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 550/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:56 2022] GET /identity/v3/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:57.031970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8dc40-b43a-4555-a6a6-d9a3819a12b0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:57.060543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:57.128998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8dc40-b43a-4555-a6a6-d9a3819a12b0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:57.162663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:57.163476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:57.163760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:57.164063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] PATH_INFO: `/v3/users/1282abf1a2b2402889d1b148890e5b69` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:57.165239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] RBAC: Authorizing `identity:get_user(user_id=1282abf1a2b2402889d1b148890e5b69)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:57.166693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:57.168345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 571/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:57 2022] GET /identity/v3/users/1282abf1a2b2402889d1b148890e5b69 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:57.174084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:57.191679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:57.270614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e466a75-778c-4a85-a99f-7836ce204ea0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:57.303528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef82309-365f-4d3f-9b00-ca9096baf2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:57.304365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef82309-365f-4d3f-9b00-ca9096baf2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:57.305160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef82309-365f-4d3f-9b00-ca9096baf2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:57.305433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef82309-365f-4d3f-9b00-ca9096baf2b4 None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:57.306621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef82309-365f-4d3f-9b00-ca9096baf2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=8fa8c2fb977e410ea9e74cffb11e6d88)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:57.308039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef82309-365f-4d3f-9b00-ca9096baf2b4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:57.309788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 551/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:26:57 2022] GET /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:57.315584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:57.408825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:57.413084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d2fc0-d70e-421a-8550-f29ef68ac614 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:57.446036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853e6fe-ec7a-4291-b6a4-0b359e34a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:57.446913 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853e6fe-ec7a-4291-b6a4-0b359e34a98c None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:57.447190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853e6fe-ec7a-4291-b6a4-0b359e34a98c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:57.447459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853e6fe-ec7a-4291-b6a4-0b359e34a98c None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88/users/1282abf1a2b2402889d1b148890e5b69/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:57.449598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853e6fe-ec7a-4291-b6a4-0b359e34a98c None admin] RBAC: Authorizing `identity:create_grant(project_id=8fa8c2fb977e410ea9e74cffb11e6d88, user_id=1282abf1a2b2402889d1b148890e5b69, role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:57.451029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853e6fe-ec7a-4291-b6a4-0b359e34a98c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:57.463020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 572/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:26:57 2022] PUT /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88/users/1282abf1a2b2402889d1b148890e5b69/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:26:59.540330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234821-0c95-43d5-b06a-4afe9cd290eb None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:59.540859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234821-0c95-43d5-b06a-4afe9cd290eb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:59.540859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5234821-0c95-43d5-b06a-4afe9cd290eb None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:59.542294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 552/1124] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:26:59.547279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8074308-fc73-4eb9-9b27-0ed2a417a818 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:59.547647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8074308-fc73-4eb9-9b27-0ed2a417a818 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:59.547841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8074308-fc73-4eb9-9b27-0ed2a417a818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:59.564482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f8074308-fc73-4eb9-9b27-0ed2a417a818 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:26:59.565648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f8074308-fc73-4eb9-9b27-0ed2a417a818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:26:59.630565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 573/1125] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:26:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:26:59.656487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5234821-0c95-43d5-b06a-4afe9cd290eb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:59.739223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:59.756247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5234821-0c95-43d5-b06a-4afe9cd290eb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:59.789083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:59.789952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:59.790236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:59.790558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] PATH_INFO: `/v3/roles/33bff0a3d0f247f6863f2e6bbe984bf9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:59.791976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] RBAC: Authorizing `identity:get_role(role_id=33bff0a3d0f247f6863f2e6bbe984bf9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:59.793415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:59.794795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 553/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:59 2022] GET /identity/v3/roles/33bff0a3d0f247f6863f2e6bbe984bf9 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:59.800545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8074308-fc73-4eb9-9b27-0ed2a417a818 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:26:59.885265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:26:59.897464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8074308-fc73-4eb9-9b27-0ed2a417a818 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:26:59.931106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:26:59.931963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:26:59.932245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:26:59.932520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] PATH_INFO: `/v3/users/1282abf1a2b2402889d1b148890e5b69` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:26:59.933701 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] RBAC: Authorizing `identity:get_user(user_id=1282abf1a2b2402889d1b148890e5b69)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:26:59.935153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:26:59.936769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 574/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:26:59 2022] GET /identity/v3/users/1282abf1a2b2402889d1b148890e5b69 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:26:59.942423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:00.037693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a827c-03d1-4027-8d13-296f1ee74069 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:00.070917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:00.071510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:00.071793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:00.072069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:00.092748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:00.137822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:00.139559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:00.240454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 554/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:26:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:27:00.247027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:00.350083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad08b47-ff52-4ad6-b380-e920c3a2186b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:00.382813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235db32-54eb-4854-81d7-b933a3fc9a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:00.383667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235db32-54eb-4854-81d7-b933a3fc9a8b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:00.383948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235db32-54eb-4854-81d7-b933a3fc9a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:00.384219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235db32-54eb-4854-81d7-b933a3fc9a8b None admin] PATH_INFO: `/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:00.385381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235db32-54eb-4854-81d7-b933a3fc9a8b None admin] RBAC: Authorizing `identity:get_project(project_id=8fa8c2fb977e410ea9e74cffb11e6d88)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:00.386829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235db32-54eb-4854-81d7-b933a3fc9a8b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:00.388512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 575/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:27:00 2022] GET /identity/v3/projects/8fa8c2fb977e410ea9e74cffb11e6d88 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:00.394576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:00.445017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:00.491169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5361363-4f15-44bb-9228-c9fe6355eda6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:00.523734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dcf1e-516b-4c48-bb4b-86fa00579935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:00.524380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dcf1e-516b-4c48-bb4b-86fa00579935 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:00.524667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dcf1e-516b-4c48-bb4b-86fa00579935 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:00.524941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dcf1e-516b-4c48-bb4b-86fa00579935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:00.525763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dcf1e-516b-4c48-bb4b-86fa00579935 None admin] RBAC: Adding query filter params (role.id=33bff0a3d0f247f6863f2e6bbe984bf9, scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88, user.id=1282abf1a2b2402889d1b148890e5b69) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:00.526105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dcf1e-516b-4c48-bb4b-86fa00579935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:00.527587 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dcf1e-516b-4c48-bb4b-86fa00579935 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:00.533291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 555/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:27:00 2022] GET /identity/v3/role_assignments?user.id=1282abf1a2b2402889d1b148890e5b69&scope.project.id=8fa8c2fb977e410ea9e74cffb11e6d88&role.id=33bff0a3d0f247f6863f2e6bbe984bf9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:02.652854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447120-e2c8-4cf0-8195-2654c9478c06 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:02.653290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447120-e2c8-4cf0-8195-2654c9478c06 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:02.653413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d447120-e2c8-4cf0-8195-2654c9478c06 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:02.654849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 576/1131] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:02.660504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002815c1-02be-4f2c-aea1-a6b9cfa16ddc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:02.660918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002815c1-02be-4f2c-aea1-a6b9cfa16ddc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:02.661056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002815c1-02be-4f2c-aea1-a6b9cfa16ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:02.684846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-002815c1-02be-4f2c-aea1-a6b9cfa16ddc None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:02.686493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-002815c1-02be-4f2c-aea1-a6b9cfa16ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:02.756088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 556/1132] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:02.780092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d447120-e2c8-4cf0-8195-2654c9478c06 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:02.805410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:02.880662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d447120-e2c8-4cf0-8195-2654c9478c06 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:02.916199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eada523-cf14-44ef-b6f1-99fb659bc2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:02.916927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada523-cf14-44ef-b6f1-99fb659bc2af None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:02.917216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada523-cf14-44ef-b6f1-99fb659bc2af None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:02.917494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada523-cf14-44ef-b6f1-99fb659bc2af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:02.918716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eada523-cf14-44ef-b6f1-99fb659bc2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:02.920138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eada523-cf14-44ef-b6f1-99fb659bc2af None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:02.921760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 577/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:27:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:02.927590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002815c1-02be-4f2c-aea1-a6b9cfa16ddc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:02.971967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:03.023057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002815c1-02be-4f2c-aea1-a6b9cfa16ddc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:03.056515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d230bfd-df3d-4ecd-84b3-4fa87c6266f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:03.057174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230bfd-df3d-4ecd-84b3-4fa87c6266f9 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:03.057461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230bfd-df3d-4ecd-84b3-4fa87c6266f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:03.057740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230bfd-df3d-4ecd-84b3-4fa87c6266f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:03.058488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d230bfd-df3d-4ecd-84b3-4fa87c6266f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:03.059914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d230bfd-df3d-4ecd-84b3-4fa87c6266f9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:03.084126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 557/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:27:02 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:27:05.210759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1093b-3861-42a5-9bd7-5e7d5d082fee None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:05.211285 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1093b-3861-42a5-9bd7-5e7d5d082fee None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:05.211375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1093b-3861-42a5-9bd7-5e7d5d082fee None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:05.212700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 578/1135] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:05.217865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c158-c6bb-433b-993c-6213c379c23d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:05.218283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c158-c6bb-433b-993c-6213c379c23d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:05.218446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c158-c6bb-433b-993c-6213c379c23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:05.235218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5027c158-c6bb-433b-993c-6213c379c23d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:05.236397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5027c158-c6bb-433b-993c-6213c379c23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:05.306567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 558/1136] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:05.330896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1093b-3861-42a5-9bd7-5e7d5d082fee None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:05.422622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:05.433004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1093b-3861-42a5-9bd7-5e7d5d082fee None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:05.466177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d13a5-c256-4fb9-b8d9-58d8b2b9e535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:05.466898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d13a5-c256-4fb9-b8d9-58d8b2b9e535 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:05.467184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d13a5-c256-4fb9-b8d9-58d8b2b9e535 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:05.467463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d13a5-c256-4fb9-b8d9-58d8b2b9e535 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:05.468611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d13a5-c256-4fb9-b8d9-58d8b2b9e535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:05.470045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d13a5-c256-4fb9-b8d9-58d8b2b9e535 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:05.471648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 579/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:27:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:05.477817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c158-c6bb-433b-993c-6213c379c23d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:05.504556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:05.574666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c158-c6bb-433b-993c-6213c379c23d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:05.607935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc373b-eb71-4bc7-a657-0ed96457b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:05.608589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc373b-eb71-4bc7-a657-0ed96457b929 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:05.608867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc373b-eb71-4bc7-a657-0ed96457b929 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:05.609134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc373b-eb71-4bc7-a657-0ed96457b929 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:05.609865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc373b-eb71-4bc7-a657-0ed96457b929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:05.611284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc373b-eb71-4bc7-a657-0ed96457b929 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:05.631720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 559/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:27:05 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:27:07.764273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037caf-1806-41e6-a834-32e63a894835 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:07.764823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037caf-1806-41e6-a834-32e63a894835 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:07.764823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037caf-1806-41e6-a834-32e63a894835 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:07.766419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 580/1139] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:07.771435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653364-6c8d-48cb-b8ad-6f1a498a26ae None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:07.771850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653364-6c8d-48cb-b8ad-6f1a498a26ae None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:07.771991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653364-6c8d-48cb-b8ad-6f1a498a26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:07.789240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fa653364-6c8d-48cb-b8ad-6f1a498a26ae None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:07.790460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fa653364-6c8d-48cb-b8ad-6f1a498a26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:07.854841 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 560/1140] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:07.878097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037caf-1806-41e6-a834-32e63a894835 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:07.974888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037caf-1806-41e6-a834-32e63a894835 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:08.007702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:08.008475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:08.008755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:08.009027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:08.018756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:27:08.026245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:27:08.027388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:27:08.028340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 581/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:27:08.033156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa653364-6c8d-48cb-b8ad-6f1a498a26ae None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:08.048065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:08.130033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa653364-6c8d-48cb-b8ad-6f1a498a26ae None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:08.165755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:08.166450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:08.166737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:08.167015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:08.167764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:08.168073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:08.169456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:08.177321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 561/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:27:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:08.184331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:08.207096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:08.282743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6f9b8-c3a2-46ff-ade3-80e06a257332 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:08.322748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:08.323874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:08.324188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:08.324469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] PATH_INFO: `/v3/users/eff9354a89e44a1d9dce67a3fbf957a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:08.339617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] RBAC: Authorizing `identity:get_user(user_id=eff9354a89e44a1d9dce67a3fbf957a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:08.341063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:08.342792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 582/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:27:08 2022] GET /identity/v3/users/eff9354a89e44a1d9dce67a3fbf957a7 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:08.348731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:08.423759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:08.446872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35340b3-1853-42df-a44a-faa5c2781fab None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:08.479402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:08.480047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:08.480327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:08.480603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:08.545824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:08.547293 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:08.618806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:08.642722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 562/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:27:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:27:08.649287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:08.725583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:08.747131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500cb87-330d-46d4-b03f-6587c8d3d26d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:08.780256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea51f6a-57b3-479a-9bcb-d64b413d0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:08.781115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea51f6a-57b3-479a-9bcb-d64b413d0f1e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:08.781406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea51f6a-57b3-479a-9bcb-d64b413d0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:08.781679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea51f6a-57b3-479a-9bcb-d64b413d0f1e None admin] PATH_INFO: `/v3/projects/4e310d75530b4092a3c0c97b5819f4d8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:08.782882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea51f6a-57b3-479a-9bcb-d64b413d0f1e None admin] RBAC: Authorizing `identity:get_project(project_id=4e310d75530b4092a3c0c97b5819f4d8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:08.784308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea51f6a-57b3-479a-9bcb-d64b413d0f1e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:08.786304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 583/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:27:08 2022] GET /identity/v3/projects/4e310d75530b4092a3c0c97b5819f4d8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:08.793069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:08.886812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0209c-49c2-4f0b-bec3-681b77afd5bd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:08.919110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b68bc-6a3f-475b-9492-8e5b1247e164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:08.919744 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b68bc-6a3f-475b-9492-8e5b1247e164 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:08.920027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b68bc-6a3f-475b-9492-8e5b1247e164 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:08.920302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b68bc-6a3f-475b-9492-8e5b1247e164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:08.921117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b68bc-6a3f-475b-9492-8e5b1247e164 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=4e310d75530b4092a3c0c97b5819f4d8, user.id=eff9354a89e44a1d9dce67a3fbf957a7) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:08.921427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b68bc-6a3f-475b-9492-8e5b1247e164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:08.922868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b68bc-6a3f-475b-9492-8e5b1247e164 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:08.928134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 563/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:27:08 2022] GET /identity/v3/role_assignments?user.id=eff9354a89e44a1d9dce67a3fbf957a7&scope.project.id=4e310d75530b4092a3c0c97b5819f4d8&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:10.993997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69cb3d-1a3a-4688-b87b-93511a0caf5c None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:10.994560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69cb3d-1a3a-4688-b87b-93511a0caf5c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:10.994560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69cb3d-1a3a-4688-b87b-93511a0caf5c None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:10.995943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 584/1147] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:11.001750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f3c4-a15e-4df0-9e21-0e075e3d951f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:11.002203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f3c4-a15e-4df0-9e21-0e075e3d951f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:11.002328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f3c4-a15e-4df0-9e21-0e075e3d951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:11.022191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-76e8f3c4-a15e-4df0-9e21-0e075e3d951f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:11.023869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-76e8f3c4-a15e-4df0-9e21-0e075e3d951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:11.057146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:11.089578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 564/1148] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:11.112617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69cb3d-1a3a-4688-b87b-93511a0caf5c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:11.211961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69cb3d-1a3a-4688-b87b-93511a0caf5c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:11.244216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:11.244948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:11.245232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:11.245509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:11.259929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:27:11.267472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:27:11.268556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:27:11.269577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 585/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:27:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:27:11.275309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f3c4-a15e-4df0-9e21-0e075e3d951f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:11.369513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f3c4-a15e-4df0-9e21-0e075e3d951f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:11.402516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:11.403183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:11.403469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:11.403745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:11.404488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:11.404796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:11.406183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:11.414279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 565/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:27:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:11.419818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:11.428962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:11.518251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac5ed1-0187-4c37-86b9-b01ef26e2412 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:11.550275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:11.551148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:11.551426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:11.551696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] PATH_INFO: `/v3/users/eff9354a89e44a1d9dce67a3fbf957a7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:11.552854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=eff9354a89e44a1d9dce67a3fbf957a7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:11.554786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:11.556376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 586/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:27:11 2022] GET /identity/v3/users/eff9354a89e44a1d9dce67a3fbf957a7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:11.562005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:11.593274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:11.661381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d79e8b-0c50-461f-b24c-abbb613264a3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:11.693669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95390b-c494-47a6-a28b-4e4f683d2bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:11.695104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95390b-c494-47a6-a28b-4e4f683d2bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:11.695390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95390b-c494-47a6-a28b-4e4f683d2bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:11.695668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95390b-c494-47a6-a28b-4e4f683d2bd8 None admin] PATH_INFO: `/v3/projects/4e310d75530b4092a3c0c97b5819f4d8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:11.696843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95390b-c494-47a6-a28b-4e4f683d2bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=4e310d75530b4092a3c0c97b5819f4d8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:11.698309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95390b-c494-47a6-a28b-4e4f683d2bd8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:11.700006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 566/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:27:11 2022] GET /identity/v3/projects/4e310d75530b4092a3c0c97b5819f4d8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:11.706071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:11.780053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:11.805526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb281a-b6c1-4003-8da8-4bcd9c015bc7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:11.838336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f4ad0-2f63-4c64-9784-c60ccb96ddb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:11.839273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f4ad0-2f63-4c64-9784-c60ccb96ddb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:11.839569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f4ad0-2f63-4c64-9784-c60ccb96ddb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:11.839846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f4ad0-2f63-4c64-9784-c60ccb96ddb7 None admin] PATH_INFO: `/v3/projects/4e310d75530b4092a3c0c97b5819f4d8/users/eff9354a89e44a1d9dce67a3fbf957a7/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:11.841781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f4ad0-2f63-4c64-9784-c60ccb96ddb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e310d75530b4092a3c0c97b5819f4d8, user_id=eff9354a89e44a1d9dce67a3fbf957a7, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:11.843223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f4ad0-2f63-4c64-9784-c60ccb96ddb7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:11.855572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 587/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:27:11 2022] PUT /identity/v3/projects/4e310d75530b4092a3c0c97b5819f4d8/users/eff9354a89e44a1d9dce67a3fbf957a7/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:27:13.957258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ba068-18b7-41b4-8e16-20358f262749 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:13.957804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ba068-18b7-41b4-8e16-20358f262749 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:13.957804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ba068-18b7-41b4-8e16-20358f262749 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:13.959139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 567/1154] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:13.964866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff22ae-7143-4e4c-8b18-93bb8956699a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:13.965261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff22ae-7143-4e4c-8b18-93bb8956699a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:13.965425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff22ae-7143-4e4c-8b18-93bb8956699a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:13.982368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-36ff22ae-7143-4e4c-8b18-93bb8956699a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:13.983545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-36ff22ae-7143-4e4c-8b18-93bb8956699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:14.050274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 588/1155] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:14.073197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ba068-18b7-41b4-8e16-20358f262749 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:14.168415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:14.172417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ba068-18b7-41b4-8e16-20358f262749 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:14.204931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:14.205707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:14.206014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:14.206288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:14.215990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:27:14.223392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:27:14.224564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:27:14.226259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 568/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:27:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:27:14.231027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff22ae-7143-4e4c-8b18-93bb8956699a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:14.296186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:14.332825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff22ae-7143-4e4c-8b18-93bb8956699a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:14.365306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:14.366022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:14.366851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:14.367125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:14.367870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:14.368175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:14.369524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:14.378620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 589/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:27:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:14.384573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:14.486071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf27fd2-25d6-4670-ac4b-54d9fe06af04 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:14.518669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:14.519541 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:14.519826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:14.520101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] PATH_INFO: `/v3/users/eff9354a89e44a1d9dce67a3fbf957a7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:14.521286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] RBAC: Authorizing `identity:get_user(user_id=eff9354a89e44a1d9dce67a3fbf957a7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:14.523243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:14.524861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 569/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:27:14 2022] GET /identity/v3/users/eff9354a89e44a1d9dce67a3fbf957a7 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:14.530695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:14.627711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a49ecd-de6b-4508-860f-2169b689f30f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:14.653408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:14.662492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:14.663747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:14.664026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:14.664295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:14.728432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:14.729873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:14.824347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 590/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:27:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:27:14.830213 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:14.859085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:14.929170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f7d8b-c793-4780-a9d3-9c0f1b2cf99e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:14.962525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab2fed-b8e0-438a-9b63-20611bdd5993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:14.963383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab2fed-b8e0-438a-9b63-20611bdd5993 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:14.963668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab2fed-b8e0-438a-9b63-20611bdd5993 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:14.963942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab2fed-b8e0-438a-9b63-20611bdd5993 None admin] PATH_INFO: `/v3/projects/4e310d75530b4092a3c0c97b5819f4d8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:14.965121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab2fed-b8e0-438a-9b63-20611bdd5993 None admin] RBAC: Authorizing `identity:get_project(project_id=4e310d75530b4092a3c0c97b5819f4d8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:14.966574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab2fed-b8e0-438a-9b63-20611bdd5993 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:14.968277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 570/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:27:14 2022] GET /identity/v3/projects/4e310d75530b4092a3c0c97b5819f4d8 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:14.974308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:15.000003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:15.077228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c85aa4-67ea-4eb9-82a3-0b9250d8c760 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:15.110522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002996f3-1447-49c8-a139-09b91e1677e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:15.111179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002996f3-1447-49c8-a139-09b91e1677e7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:15.111464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002996f3-1447-49c8-a139-09b91e1677e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:15.111739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002996f3-1447-49c8-a139-09b91e1677e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:15.112572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002996f3-1447-49c8-a139-09b91e1677e7 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=4e310d75530b4092a3c0c97b5819f4d8, user.id=eff9354a89e44a1d9dce67a3fbf957a7) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:15.112920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002996f3-1447-49c8-a139-09b91e1677e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:15.114386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002996f3-1447-49c8-a139-09b91e1677e7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:15.120320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 591/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:27:14 2022] GET /identity/v3/role_assignments?user.id=eff9354a89e44a1d9dce67a3fbf957a7&scope.project.id=4e310d75530b4092a3c0c97b5819f4d8&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:17.226790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd871ac-5175-4b2e-b448-db4e204a4d78 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:17.227338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd871ac-5175-4b2e-b448-db4e204a4d78 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:17.227338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd871ac-5175-4b2e-b448-db4e204a4d78 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:17.228700 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 571/1162] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:17.233908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b39cb-bba4-48ec-bb10-6fd95c31eeb6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:17.234348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b39cb-bba4-48ec-bb10-6fd95c31eeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:17.234449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b39cb-bba4-48ec-bb10-6fd95c31eeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:17.250718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-444b39cb-bba4-48ec-bb10-6fd95c31eeb6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:17.251896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-444b39cb-bba4-48ec-bb10-6fd95c31eeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:17.315853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 592/1163] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:17.338993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd871ac-5175-4b2e-b448-db4e204a4d78 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:17.431601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:17.438661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd871ac-5175-4b2e-b448-db4e204a4d78 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:17.471636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:17.472277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:17.472564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:17.472843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:17.537830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:17.539287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:17.628609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:17.636224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 572/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:27:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:27:17.642050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b39cb-bba4-48ec-bb10-6fd95c31eeb6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:17.662566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:17.742534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b39cb-bba4-48ec-bb10-6fd95c31eeb6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:17.775271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00b58-237f-4fd8-bc07-4b4c504aa1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:17.776065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00b58-237f-4fd8-bc07-4b4c504aa1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:17.776352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00b58-237f-4fd8-bc07-4b4c504aa1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:17.776623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00b58-237f-4fd8-bc07-4b4c504aa1a1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:17.784353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00b58-237f-4fd8-bc07-4b4c504aa1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:17.786054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00b58-237f-4fd8-bc07-4b4c504aa1a1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-98e00b58-237f-4fd8-bc07-4b4c504aa1a1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:27:17.799753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:27:17.800981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:27:17.802280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 593/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:27:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:27:17.807387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:17.903060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fe70f-e473-4ee6-8685-f11dd97e288b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:17.937025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a291-c566-40fd-bafd-3a96f62f2f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:17.937677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a291-c566-40fd-bafd-3a96f62f2f84 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:17.937992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a291-c566-40fd-bafd-3a96f62f2f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:17.938270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a291-c566-40fd-bafd-3a96f62f2f84 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:17.938987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a291-c566-40fd-bafd-3a96f62f2f84 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:17.939293 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a291-c566-40fd-bafd-3a96f62f2f84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:17.940648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a291-c566-40fd-bafd-3a96f62f2f84 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:17.947631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 573/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Fri Jun 17 11:27:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:20.019410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788bb5b-3fb3-4953-9a9f-cb1817db60b7 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:20.020000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788bb5b-3fb3-4953-9a9f-cb1817db60b7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:20.020000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788bb5b-3fb3-4953-9a9f-cb1817db60b7 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:20.021329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 594/1167] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:20.026381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d394c9-aa40-4f06-9913-3380d8721e1f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:20.026751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d394c9-aa40-4f06-9913-3380d8721e1f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:20.026943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d394c9-aa40-4f06-9913-3380d8721e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:20.043534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-19d394c9-aa40-4f06-9913-3380d8721e1f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:20.044749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-19d394c9-aa40-4f06-9913-3380d8721e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:20.099330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:20.110666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 574/1168] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:20.134014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788bb5b-3fb3-4953-9a9f-cb1817db60b7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:20.213980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:20.234355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788bb5b-3fb3-4953-9a9f-cb1817db60b7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:20.267436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573790f8-3b55-4eb8-9c91-194f683b212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:20.268180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573790f8-3b55-4eb8-9c91-194f683b212f None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:20.268472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573790f8-3b55-4eb8-9c91-194f683b212f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:20.268753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573790f8-3b55-4eb8-9c91-194f683b212f None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:20.269536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573790f8-3b55-4eb8-9c91-194f683b212f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:20.270884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573790f8-3b55-4eb8-9c91-194f683b212f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:20.286190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 595/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Fri Jun 17 11:27:20 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:27:22.438695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d790285-63f9-4e94-b3b2-887767d6ede2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:22.439596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d790285-63f9-4e94-b3b2-887767d6ede2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:22.439596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d790285-63f9-4e94-b3b2-887767d6ede2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:22.440619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 575/1170] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:22.446517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb436-955d-480c-9874-7043671ffe90 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:22.446958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb436-955d-480c-9874-7043671ffe90 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:22.447061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bb436-955d-480c-9874-7043671ffe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:22.465473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-670bb436-955d-480c-9874-7043671ffe90 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:22.466705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-670bb436-955d-480c-9874-7043671ffe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:22.531877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 596/1171] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:22.555771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d790285-63f9-4e94-b3b2-887767d6ede2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:22.653992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d790285-63f9-4e94-b3b2-887767d6ede2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:22.686670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca0b86-66cf-4926-aaf9-f235368846bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:22.687518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0b86-66cf-4926-aaf9-f235368846bf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:22.687801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0b86-66cf-4926-aaf9-f235368846bf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:22.688070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca0b86-66cf-4926-aaf9-f235368846bf None admin] PATH_INFO: `/v3/domains/66c0a5ffadc245b0b959df97d77e202d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:22.698634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca0b86-66cf-4926-aaf9-f235368846bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=66c0a5ffadc245b0b959df97d77e202d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:22.700013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca0b86-66cf-4926-aaf9-f235368846bf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:22.701634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 576/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Fri Jun 17 11:27:22 2022] GET /identity/v3/domains/66c0a5ffadc245b0b959df97d77e202d => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:22.707694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bb436-955d-480c-9874-7043671ffe90 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:22.776249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:22.807235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bb436-955d-480c-9874-7043671ffe90 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:22.842208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21e6f8-314f-4eaa-8e81-28a0305b8b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:22.842862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e6f8-314f-4eaa-8e81-28a0305b8b03 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:22.843143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e6f8-314f-4eaa-8e81-28a0305b8b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:22.843417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e6f8-314f-4eaa-8e81-28a0305b8b03 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:22.844113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21e6f8-314f-4eaa-8e81-28a0305b8b03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:22.845533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21e6f8-314f-4eaa-8e81-28a0305b8b03 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:22.869702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 597/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:27:22 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:27:25.000972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3eadd-f758-45d6-855e-2654592e82d5 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:25.001512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3eadd-f758-45d6-855e-2654592e82d5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:25.001512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3eadd-f758-45d6-855e-2654592e82d5 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:25.002940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 577/1174] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:25.008187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9ea98-d594-4d98-b7da-71e7662dec73 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:25.008634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9ea98-d594-4d98-b7da-71e7662dec73 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:25.008740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9ea98-d594-4d98-b7da-71e7662dec73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:25.026258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a7f9ea98-d594-4d98-b7da-71e7662dec73 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:25.027429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a7f9ea98-d594-4d98-b7da-71e7662dec73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:25.091401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 598/1175] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:25.116242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3eadd-f758-45d6-855e-2654592e82d5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:25.166419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:25.215067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3eadd-f758-45d6-855e-2654592e82d5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:25.247431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414da70-0aa4-452f-94d8-4ab0f8611627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:25.248276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414da70-0aa4-452f-94d8-4ab0f8611627 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:25.248559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414da70-0aa4-452f-94d8-4ab0f8611627 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:25.248834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414da70-0aa4-452f-94d8-4ab0f8611627 None admin] PATH_INFO: `/v3/domains/66c0a5ffadc245b0b959df97d77e202d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:25.250023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414da70-0aa4-452f-94d8-4ab0f8611627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=66c0a5ffadc245b0b959df97d77e202d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:25.251504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414da70-0aa4-452f-94d8-4ab0f8611627 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:25.253048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 578/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Fri Jun 17 11:27:25 2022] GET /identity/v3/domains/66c0a5ffadc245b0b959df97d77e202d => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:25.258630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9ea98-d594-4d98-b7da-71e7662dec73 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:25.309047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:25.354475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9ea98-d594-4d98-b7da-71e7662dec73 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:25.387189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9407e-cac3-48ec-a489-f97123fdfb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:25.387852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9407e-cac3-48ec-a489-f97123fdfb5e None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:25.388135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9407e-cac3-48ec-a489-f97123fdfb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:25.388414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9407e-cac3-48ec-a489-f97123fdfb5e None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:25.389124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9407e-cac3-48ec-a489-f97123fdfb5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:25.390596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9407e-cac3-48ec-a489-f97123fdfb5e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:25.410463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 599/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:27:25 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:27:27.564862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943df1a-3bae-4d3c-8357-21d994089ca7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:27.565421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943df1a-3bae-4d3c-8357-21d994089ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:27.565421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943df1a-3bae-4d3c-8357-21d994089ca7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:27.566808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 579/1178] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:27.572590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21928b-a68a-4c20-9754-f7bc62771ee1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:27.572971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21928b-a68a-4c20-9754-f7bc62771ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:27.573154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21928b-a68a-4c20-9754-f7bc62771ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:27.589506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1a21928b-a68a-4c20-9754-f7bc62771ee1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:27.590728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1a21928b-a68a-4c20-9754-f7bc62771ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:27.656481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 600/1179] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:27.681220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943df1a-3bae-4d3c-8357-21d994089ca7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:27.786533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943df1a-3bae-4d3c-8357-21d994089ca7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:27.804107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:27.820548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:27.821278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:27.821561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:27.821852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:27.831718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:27:27.839304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:27:27.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:27:27.841392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 580/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:27:27.846518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21928b-a68a-4c20-9754-f7bc62771ee1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:27.885723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:27.946837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21928b-a68a-4c20-9754-f7bc62771ee1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:27.979897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:27.980554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:27.980835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:27.981114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:27.981897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:27.982194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:27.983547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:27.992161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 601/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:27.998050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:28.095473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968891b6-1066-47af-a5fe-85218c2afe95 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:28.127655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:28.128514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:28.128795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:28.129071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] PATH_INFO: `/v3/users/2204ffb44af44f34b8450bb56d782d71` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:28.141781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] RBAC: Authorizing `identity:get_user(user_id=2204ffb44af44f34b8450bb56d782d71)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:28.143252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:28.144879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 581/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:27:27 2022] GET /identity/v3/users/2204ffb44af44f34b8450bb56d782d71 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:28.151084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:28.246272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:28.249294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713843f-ee6a-4175-b87c-df7ffa8a6958 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:28.282401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:28.283047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:28.283335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:28.283621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:28.349440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:28.351459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:28.447823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:28.451729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 602/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:27:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:27:28.457557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:28.492090 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:28.555732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab84cfe-00a2-44f3-8010-7e619d4edbd6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:28.588861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c366c44-d8a7-4abb-b561-114c196c4df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:28.589710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c366c44-d8a7-4abb-b561-114c196c4df0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:28.590023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c366c44-d8a7-4abb-b561-114c196c4df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:28.590302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c366c44-d8a7-4abb-b561-114c196c4df0 None admin] PATH_INFO: `/v3/projects/6d879109352e44c9a656de647e438738` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:28.591550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c366c44-d8a7-4abb-b561-114c196c4df0 None admin] RBAC: Authorizing `identity:get_project(project_id=6d879109352e44c9a656de647e438738)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:28.592977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c366c44-d8a7-4abb-b561-114c196c4df0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:28.594693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 582/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:27:28 2022] GET /identity/v3/projects/6d879109352e44c9a656de647e438738 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:28.600836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:28.697631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636cdf5-2a7f-4bd4-a825-a62443bea4e2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:28.730917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930eece2-d1e6-467f-8b62-3f90941b90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:28.731573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eece2-d1e6-467f-8b62-3f90941b90f3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:28.731859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eece2-d1e6-467f-8b62-3f90941b90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:28.732135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930eece2-d1e6-467f-8b62-3f90941b90f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:28.732969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eece2-d1e6-467f-8b62-3f90941b90f3 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=6d879109352e44c9a656de647e438738, user.id=2204ffb44af44f34b8450bb56d782d71) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:28.733285 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eece2-d1e6-467f-8b62-3f90941b90f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:28.734729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930eece2-d1e6-467f-8b62-3f90941b90f3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:28.740084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 603/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:27:28 2022] GET /identity/v3/role_assignments?user.id=2204ffb44af44f34b8450bb56d782d71&scope.project.id=6d879109352e44c9a656de647e438738&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:30.820116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434503f-3078-46ad-9eba-cf904fadec2e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:30.820630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434503f-3078-46ad-9eba-cf904fadec2e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:30.820741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434503f-3078-46ad-9eba-cf904fadec2e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:30.822080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 583/1186] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:30.827134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785069e6-080c-4a12-ab23-50de05364e33 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:30.827561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785069e6-080c-4a12-ab23-50de05364e33 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:30.827687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785069e6-080c-4a12-ab23-50de05364e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:30.843944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-785069e6-080c-4a12-ab23-50de05364e33 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:30.845109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-785069e6-080c-4a12-ab23-50de05364e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:30.903086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:30.911550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 604/1187] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:30.934635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434503f-3078-46ad-9eba-cf904fadec2e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:31.032238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434503f-3078-46ad-9eba-cf904fadec2e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:31.034756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:31.066285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:31.067065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:31.067350 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:31.067624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:31.077505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:27:31.085500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:27:31.086642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:27:31.087603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 584/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:27:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:27:31.093134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785069e6-080c-4a12-ab23-50de05364e33 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:31.188887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785069e6-080c-4a12-ab23-50de05364e33 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:31.221417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:31.222106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:31.222393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:31.222666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:31.223408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:31.223717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:31.225686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:31.233497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 605/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:27:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:31.239152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:31.334955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5cd4c-99ac-49be-beb1-ebf4861112fe None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:31.367274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:31.368133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:31.368419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:31.368696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] PATH_INFO: `/v3/users/2204ffb44af44f34b8450bb56d782d71` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:31.369897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] RBAC: Authorizing `identity:get_user(user_id=2204ffb44af44f34b8450bb56d782d71)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:31.371330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:31.372943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 585/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:27:31 2022] GET /identity/v3/users/2204ffb44af44f34b8450bb56d782d71 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:31.378637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:31.433760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:31.477257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485834d1-6f96-4175-abcd-9f6152ca7080 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:31.509422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec7875-3754-49ea-80c1-66700ca3e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:31.510307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec7875-3754-49ea-80c1-66700ca3e1ae None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:31.510592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec7875-3754-49ea-80c1-66700ca3e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:31.510869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec7875-3754-49ea-80c1-66700ca3e1ae None admin] PATH_INFO: `/v3/projects/6d879109352e44c9a656de647e438738` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:31.512036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec7875-3754-49ea-80c1-66700ca3e1ae None admin] RBAC: Authorizing `identity:get_project(project_id=6d879109352e44c9a656de647e438738)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:31.513455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec7875-3754-49ea-80c1-66700ca3e1ae None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:31.515173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 606/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:27:31 2022] GET /identity/v3/projects/6d879109352e44c9a656de647e438738 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:31.520958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:31.552101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:31.618811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a57fa6-71b1-4110-b333-a3810efb55fb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:31.652529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4c783-cac8-4bad-b771-fee74a9f9be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:31.653411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4c783-cac8-4bad-b771-fee74a9f9be9 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:31.653693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4c783-cac8-4bad-b771-fee74a9f9be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:31.654004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4c783-cac8-4bad-b771-fee74a9f9be9 None admin] PATH_INFO: `/v3/projects/6d879109352e44c9a656de647e438738/users/2204ffb44af44f34b8450bb56d782d71/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:31.655954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4c783-cac8-4bad-b771-fee74a9f9be9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d879109352e44c9a656de647e438738, user_id=2204ffb44af44f34b8450bb56d782d71, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:31.657375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4c783-cac8-4bad-b771-fee74a9f9be9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:31.671362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 586/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:27:31 2022] PUT /identity/v3/projects/6d879109352e44c9a656de647e438738/users/2204ffb44af44f34b8450bb56d782d71/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:27:33.770387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966665d-d0c9-4753-b35d-19b0943c1888 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:33.770931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966665d-d0c9-4753-b35d-19b0943c1888 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:33.770931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966665d-d0c9-4753-b35d-19b0943c1888 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:33.772357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 607/1193] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:27:33.777505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968da5-d966-46f7-9c5a-ad325cd35b38 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:33.777950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968da5-d966-46f7-9c5a-ad325cd35b38 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:33.778109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c968da5-d966-46f7-9c5a-ad325cd35b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:33.795161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0c968da5-d966-46f7-9c5a-ad325cd35b38 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:27:33.796347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0c968da5-d966-46f7-9c5a-ad325cd35b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:27:33.861611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 587/1194] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:27:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:27:33.885137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966665d-d0c9-4753-b35d-19b0943c1888 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:33.985768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966665d-d0c9-4753-b35d-19b0943c1888 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:34.008389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:34.019011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:34.019798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:34.020085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:34.020364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:34.031349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:27:34.039696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:27:34.040801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:27:34.042046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 608/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:27:34.046830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c968da5-d966-46f7-9c5a-ad325cd35b38 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:34.069868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:34.143495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c968da5-d966-46f7-9c5a-ad325cd35b38 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:34.177608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:34.178307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:34.178594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:34.178871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:34.179617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:34.179924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:34.181287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:34.189117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 588/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:27:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:34.194926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:34.296513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecaef40-d954-400f-a4c5-de7cd525f05f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:34.328984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:34.329867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:34.330160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:34.330435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] PATH_INFO: `/v3/users/2204ffb44af44f34b8450bb56d782d71` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:34.331616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] RBAC: Authorizing `identity:get_user(user_id=2204ffb44af44f34b8450bb56d782d71)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:34.333043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:34.335223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 609/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Fri Jun 17 11:27:34 2022] GET /identity/v3/users/2204ffb44af44f34b8450bb56d782d71 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:34.341191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:34.424490 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:34.439982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad4c43-59b5-417c-beca-2909d87b09e6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:34.472711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:34.473498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:34.473788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:34.474210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:34.542946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:34.544392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:34.624024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:34.641295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 589/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:27:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:27:34.647688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:34.699409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:34.745791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05b95a-1795-4f4e-a133-99f9eaddf997 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:34.779558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3450b2-40db-4c2d-8035-c8583d5bf99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:34.780416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3450b2-40db-4c2d-8035-c8583d5bf99b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:34.780700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3450b2-40db-4c2d-8035-c8583d5bf99b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:34.780977 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3450b2-40db-4c2d-8035-c8583d5bf99b None admin] PATH_INFO: `/v3/projects/6d879109352e44c9a656de647e438738` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:34.782185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3450b2-40db-4c2d-8035-c8583d5bf99b None admin] RBAC: Authorizing `identity:get_project(project_id=6d879109352e44c9a656de647e438738)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:34.783610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3450b2-40db-4c2d-8035-c8583d5bf99b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:34.785557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 610/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Fri Jun 17 11:27:34 2022] GET /identity/v3/projects/6d879109352e44c9a656de647e438738 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:34.791576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:27:34.887810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180982-f64b-4815-8760-84dfe0f7e334 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:27:34.921165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceba61f-b111-4db8-8875-1bcef5550eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:27:34.921815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba61f-b111-4db8-8875-1bcef5550eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:27:34.922127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba61f-b111-4db8-8875-1bcef5550eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:27:34.922404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceba61f-b111-4db8-8875-1bcef5550eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:27:34.923233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceba61f-b111-4db8-8875-1bcef5550eb6 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=6d879109352e44c9a656de647e438738, user.id=2204ffb44af44f34b8450bb56d782d71) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:27:34.923548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceba61f-b111-4db8-8875-1bcef5550eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:27:34.924969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceba61f-b111-4db8-8875-1bcef5550eb6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:27:34.931329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 590/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:27:34 2022] GET /identity/v3/role_assignments?user.id=2204ffb44af44f34b8450bb56d782d71&scope.project.id=6d879109352e44c9a656de647e438738&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:27:44.775471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=40 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:27:44.928801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=76 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:31.275709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe510282-5ba1-4f46-9f0c-7133b866efe9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:31.276364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe510282-5ba1-4f46-9f0c-7133b866efe9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:31.276364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe510282-5ba1-4f46-9f0c-7133b866efe9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:31.277644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 611/1201] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:28:31.284270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69836c-1502-46f8-a453-18ba5cd2a22c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:31.284795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69836c-1502-46f8-a453-18ba5cd2a22c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:31.284894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69836c-1502-46f8-a453-18ba5cd2a22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:31.308092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1a69836c-1502-46f8-a453-18ba5cd2a22c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:28:31.309411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1a69836c-1502-46f8-a453-18ba5cd2a22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:28:31.402653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 591/1202] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:28:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:28:31.433703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe510282-5ba1-4f46-9f0c-7133b866efe9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:31.576487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe510282-5ba1-4f46-9f0c-7133b866efe9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:31.633306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:31.634481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:31.634588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:31.634849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:31.648916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:31.650676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:31.674814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:31.676215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:31.677202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:31.677202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:31.677202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:31.677202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:31.684824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 612/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:31.685002 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69836c-1502-46f8-a453-18ba5cd2a22c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:31.813030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69836c-1502-46f8-a453-18ba5cd2a22c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:31.853954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:31.860004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bedfdb-4721-4dd7-a485-4d2490844008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:31.860754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bedfdb-4721-4dd7-a485-4d2490844008 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:31.861097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bedfdb-4721-4dd7-a485-4d2490844008 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:31.861430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bedfdb-4721-4dd7-a485-4d2490844008 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:31.862820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bedfdb-4721-4dd7-a485-4d2490844008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:31.863163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bedfdb-4721-4dd7-a485-4d2490844008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:31.864581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bedfdb-4721-4dd7-a485-4d2490844008 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:31.876769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 592/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:31.887318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:31.965751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:32.021764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f631ab-ac8c-42ab-9cf3-2677b4987da7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:32.065610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9eb3-5afc-41ed-9a7d-b4c70a8b412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:32.066879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9eb3-5afc-41ed-9a7d-b4c70a8b412a None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:32.067240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9eb3-5afc-41ed-9a7d-b4c70a8b412a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:32.067613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9eb3-5afc-41ed-9a7d-b4c70a8b412a None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:32.068470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f9eb3-5afc-41ed-9a7d-b4c70a8b412a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:32.070231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f9eb3-5afc-41ed-9a7d-b4c70a8b412a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:32.103271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 613/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Fri Jun 17 11:28:31 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:28:35.007658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd946ee8-8152-4e03-8b82-38d97060eea1 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:35.008205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd946ee8-8152-4e03-8b82-38d97060eea1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:35.008638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd946ee8-8152-4e03-8b82-38d97060eea1 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:35.010605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 593/1206] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:28:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:28:35.020056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925006f9-92d4-493b-a934-96da7f4f4e7f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:35.020488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925006f9-92d4-493b-a934-96da7f4f4e7f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:35.020611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925006f9-92d4-493b-a934-96da7f4f4e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:35.038873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-925006f9-92d4-493b-a934-96da7f4f4e7f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:28:35.040177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-925006f9-92d4-493b-a934-96da7f4f4e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:28:35.121577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 614/1207] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:28:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:28:35.150389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd946ee8-8152-4e03-8b82-38d97060eea1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:35.269960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd946ee8-8152-4e03-8b82-38d97060eea1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:35.311828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:35.313031 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:35.313539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:35.314066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:35.329116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:28:35.337379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:35.343866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:35.344747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 594/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:28:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:35.345484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925006f9-92d4-493b-a934-96da7f4f4e7f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:35.395427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:35.464939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925006f9-92d4-493b-a934-96da7f4f4e7f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:35.503850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:35.504599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:35.504941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:35.505217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:35.506175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:35.506484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:35.508173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:35.523981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 615/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:28:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:35.525588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:35.591707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:35.648306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7851468-ee27-4f5b-bb98-4834003eb385 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:35.687979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:35.689130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:35.689393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:35.689685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:35.702183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:35.703702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:35.723077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:35.724890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:35.726174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:35.726174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:35.726174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:35.726174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:35.730966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 595/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:35.731118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:35.837205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:35.840558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7ba89-7671-4e56-877a-8aadc8505e07 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:35.876767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:35.878149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:35.878381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:35.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:35.879386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:35.879696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:35.881067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:35.893825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 616/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:35.899245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:36.009425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ff6ea-2be6-4310-b2a6-d566c0942473 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:36.011014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:36.044648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:36.045486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:36.045776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:36.046086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:36.054626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:36.056223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:28:36.071368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:36.072790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:36.073762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 596/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:28:36.078598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:36.186655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b87778-7ca9-44c8-8698-8f38dee44f03 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:36.225344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:36.226063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:36.226348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:36.226621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:36.228541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:36.228893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:36.230598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:36.257875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 617/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Jun 17 11:28:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:36.259113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:36.375092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e270c7-401c-4f6b-955b-4479aab711a6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:36.413832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:36.414631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:36.414990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:36.415333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:36.449396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:36.498649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:36.500111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:36.617334 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 597/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:28:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:28:36.623158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:36.653161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:36.755211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84a5a8-d79f-4205-b83d-d9c2c8b150d2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:36.793296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:36.794195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:36.794483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:36.794760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:36.810116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:36.811784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:36.836293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:36.837494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:36.838708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:36.838708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:36.838708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:36.838708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:36.838708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 618/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:36.843516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:36.914302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:36.958366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e914a9b-9c99-4ceb-9e4e-8dc63392f458 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:36.994376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:36.995029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:36.995310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:36.995583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:36.996300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:36.996605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:36.997994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:37.011716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 598/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:37.017637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:37.106812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:37.146862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb537f-1de2-49f0-97a9-19862764ca98 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:37.188361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:37.189290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:37.189648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:37.190588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:37.197679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:37.199165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:28:37.213777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:37.215011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:37.215963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 619/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:28:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:37.223431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:37.337522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04226ef3-3959-41bb-ab30-f44ed9d08143 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:37.364138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:37.380926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa318df-4dbc-41bb-9d79-1ca2cf1447bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:37.381734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa318df-4dbc-41bb-9d79-1ca2cf1447bb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:37.382156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa318df-4dbc-41bb-9d79-1ca2cf1447bb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:37.382442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa318df-4dbc-41bb-9d79-1ca2cf1447bb None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:37.383288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa318df-4dbc-41bb-9d79-1ca2cf1447bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:37.383600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa318df-4dbc-41bb-9d79-1ca2cf1447bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:37.385094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa318df-4dbc-41bb-9d79-1ca2cf1447bb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:37.402555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 599/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:28:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:37.409124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:37.526158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb774b-caa1-483e-8573-62f8d4d30e38 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:37.559512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:37.566139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce28d85-4fad-43f1-a5c9-3985fcfa4b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:37.566923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce28d85-4fad-43f1-a5c9-3985fcfa4b24 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:37.567311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce28d85-4fad-43f1-a5c9-3985fcfa4b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:37.567725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce28d85-4fad-43f1-a5c9-3985fcfa4b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:37.568870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce28d85-4fad-43f1-a5c9-3985fcfa4b24 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=1c8bdac27bde4a2785d4aaad8213baca) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:37.569334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce28d85-4fad-43f1-a5c9-3985fcfa4b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:37.570965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce28d85-4fad-43f1-a5c9-3985fcfa4b24 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:37.577742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 620/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:28:37 2022] GET /identity/v3/role_assignments?user.id=1c8bdac27bde4a2785d4aaad8213baca&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:39.871974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184deb86-14d9-4487-a4ea-d4d2f4222481 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:39.872521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184deb86-14d9-4487-a4ea-d4d2f4222481 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:39.872521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184deb86-14d9-4487-a4ea-d4d2f4222481 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:39.873911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 600/1220] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:28:39.880160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48035faf-3b34-4639-80aa-fcbf1ef21b54 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:39.880595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48035faf-3b34-4639-80aa-fcbf1ef21b54 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:39.880696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48035faf-3b34-4639-80aa-fcbf1ef21b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:39.898799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-48035faf-3b34-4639-80aa-fcbf1ef21b54 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:28:39.900188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-48035faf-3b34-4639-80aa-fcbf1ef21b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:28:39.972212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 621/1221] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:28:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:28:39.998261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184deb86-14d9-4487-a4ea-d4d2f4222481 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:40.108756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184deb86-14d9-4487-a4ea-d4d2f4222481 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:40.147102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:40.147909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:40.148195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:40.148470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:40.159497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:28:40.167166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:40.168347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:40.169266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 601/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:28:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:40.175343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48035faf-3b34-4639-80aa-fcbf1ef21b54 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:40.286585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48035faf-3b34-4639-80aa-fcbf1ef21b54 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:40.310687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:40.324762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:40.325736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:40.326134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:40.326506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:40.327324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:40.327669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:40.329359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:40.338256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 622/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:28:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:40.343989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:40.395138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:40.462958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74736da7-28f7-47ef-8de0-be91fc6e99c6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:40.498799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:40.499590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:40.499900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:40.500167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:40.511156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:40.512620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:40.533171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:40.535044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:40.536460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:40.536460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:40.536460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:40.536460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:40.536460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 602/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:40.540756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:40.649621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de238c93-f766-4055-a19d-c7263e66d53d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:40.685495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:40.686196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:40.686486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:40.686759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:40.687516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:40.687826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:40.689192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:40.700920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 623/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:40.706595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:40.816583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:40.823627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8345e05-bfb7-4376-87be-2facdf5488b8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:40.863689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:40.864670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:40.865014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:40.865301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:40.874629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:40.876497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:28:40.894123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:40.895307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:40.896271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 603/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:40.901338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:40.959296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:41.027325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4761e4a-c3a5-42ae-89a6-321d2aafe329 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:41.069386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:41.070158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:41.070378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:41.070691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:41.071681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:41.072224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:41.074291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:41.094440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 624/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Jun 17 11:28:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:41.100998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:41.208062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b85a-5445-4930-8b9e-2a1d4b9fd4a3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:41.243681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:41.244484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:41.244765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:41.245038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:41.256129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:41.257573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:41.275015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:41.276044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:41.278825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:41.278825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:41.278825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:41.278825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:41.278825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 604/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:28:41.281986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:41.401305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18a31-2b5a-4cd4-b4d8-79394a2c38a5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:41.403825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:41.437146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:41.437944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:41.438307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:41.438587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:41.439315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:41.440155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:41.441520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:41.460283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 625/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:41.461538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:41.471073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:41.569180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217af64f-1557-4681-ad07-b6a124bcd08c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:41.603713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:41.604515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:41.604802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:41.605079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:41.612414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:41.613882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:28:41.628747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:41.629893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:41.631026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 605/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:28:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:28:41.637539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:41.747928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97225161-da08-4aae-ad0b-cc913719dc3e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:41.783553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e5dc7-07a3-4507-8203-8a2b97ce84e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:41.784318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5dc7-07a3-4507-8203-8a2b97ce84e2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:41.784676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5dc7-07a3-4507-8203-8a2b97ce84e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:41.785024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5dc7-07a3-4507-8203-8a2b97ce84e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:41.785920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e5dc7-07a3-4507-8203-8a2b97ce84e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:41.786307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e5dc7-07a3-4507-8203-8a2b97ce84e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:41.787802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e5dc7-07a3-4507-8203-8a2b97ce84e2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:41.800912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 626/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:41.806948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:41.878945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:41.917225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529ab2c-6edd-4669-8b56-d8b937e94b18 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:41.966332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f0bbf-1335-4bff-8a90-a36f28b39bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:41.967438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f0bbf-1335-4bff-8a90-a36f28b39bd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:41.967714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f0bbf-1335-4bff-8a90-a36f28b39bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:41.968073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f0bbf-1335-4bff-8a90-a36f28b39bd9 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/1c8bdac27bde4a2785d4aaad8213baca/roles/b3c5094c32f446d88c32145d8b8ae3cd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:41.985195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f0bbf-1335-4bff-8a90-a36f28b39bd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=1c8bdac27bde4a2785d4aaad8213baca, role_id=b3c5094c32f446d88c32145d8b8ae3cd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:41.987307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f0bbf-1335-4bff-8a90-a36f28b39bd9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:42.000800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 606/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:28:41 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/1c8bdac27bde4a2785d4aaad8213baca/roles/b3c5094c32f446d88c32145d8b8ae3cd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:28:44.136505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c49b00-df66-41b6-afd9-92bff0d19b07 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:44.137053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c49b00-df66-41b6-afd9-92bff0d19b07 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:44.137148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c49b00-df66-41b6-afd9-92bff0d19b07 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:44.139052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 627/1233] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:28:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:28:44.144862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45d455-0e1d-47f9-85ef-a5591ab786f8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:44.145291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45d455-0e1d-47f9-85ef-a5591ab786f8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:44.145426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45d455-0e1d-47f9-85ef-a5591ab786f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:44.162519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ac45d455-0e1d-47f9-85ef-a5591ab786f8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:28:44.163741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ac45d455-0e1d-47f9-85ef-a5591ab786f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:28:44.237889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 607/1234] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:28:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:28:44.261376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c49b00-df66-41b6-afd9-92bff0d19b07 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:44.290341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:44.370094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c49b00-df66-41b6-afd9-92bff0d19b07 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:44.406388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:44.407658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:44.407918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:44.408279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:44.418688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:28:44.427596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:44.428749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:44.429767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 628/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Fri Jun 17 11:28:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:44.435580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45d455-0e1d-47f9-85ef-a5591ab786f8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:44.489948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:44.552247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45d455-0e1d-47f9-85ef-a5591ab786f8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:44.591494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:44.592241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:44.592633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:44.592911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:44.593654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:44.595162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:44.595351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:44.604649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 608/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Fri Jun 17 11:28:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:44.610464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:44.702517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:44.722118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812da57-9efc-4a1f-92fa-767b9824c709 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:44.764263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:44.765169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:44.765459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:44.765851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:44.777439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:44.778967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:44.796844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:44.797944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:44.799388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:44.799388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:44.799388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:44.799388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:44.799388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 629/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:44.804585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:44.911569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de21c-9cf0-4083-b838-e555e098b8ec None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:44.922677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:44.950487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:44.951158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:44.951441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:44.951717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:44.952437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:44.952797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:44.955002 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:44.966843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 609/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:44.973246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:45.089756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f51bf-e3e8-44c2-86ba-248bb550a76b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:45.128145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:45.128951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:45.129235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:45.129509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:45.132352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:45.138925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:45.141200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:28:45.156289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:45.158502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:45.159676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:45.163176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 630/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:45.164090 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:45.274488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce31b9-c41e-4608-b636-b9fc0faed502 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:45.316869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:45.317516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:45.317795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:45.318131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:45.319017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:45.319289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:45.320711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:45.338734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 610/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Jun 17 11:28:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:45.344501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:45.465694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e133a6b-0ca4-4273-a9b5-83cd0c8fe5c8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:45.501296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:45.502062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:45.502405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:45.502628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:45.572755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:45.579357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:45.581925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:45.694466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 631/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:28:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:28:45.701153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:45.730554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:45.811851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce031-517b-4fca-99ce-19696ba911d5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:45.849037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:45.849742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:45.850046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:45.850319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:45.860750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:45.862459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:45.880327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:45.881438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:45.882489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:45.882489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:45.882489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:45.882489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:45.882489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 611/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:45.887701 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:45.987495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:45.994447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245992c-ba92-4bb6-a150-be46765bb3d3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:46.031802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:46.032746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:46.033059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:46.033342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:46.034112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:46.034424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:46.035803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:46.049516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 632/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:46.055634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:46.144529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:46.170766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112661a-a24b-467f-8ff6-525560a9e8bc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:46.208285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:46.209296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:46.209582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:46.209959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:46.219026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:46.220648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:28:46.240102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:46.241309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:46.242301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:46.249753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 612/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:28:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:46.252813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:46.388435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0fe4a-2f3c-4377-8b77-e4d0a86f0db2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:46.429552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc68e1-731c-4d76-9c31-b985bbfaddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:46.430474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc68e1-731c-4d76-9c31-b985bbfaddcc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:46.430794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc68e1-731c-4d76-9c31-b985bbfaddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:46.431202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc68e1-731c-4d76-9c31-b985bbfaddcc None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:46.432228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc68e1-731c-4d76-9c31-b985bbfaddcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:46.432590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc68e1-731c-4d76-9c31-b985bbfaddcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:46.435365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc68e1-731c-4d76-9c31-b985bbfaddcc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:46.451123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 633/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:28:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:46.458355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:46.586349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617196e6-cab1-45e0-8f6e-5522789c031c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:46.628124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c1910-0a46-4cb4-998d-3565fd698179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:46.628818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1910-0a46-4cb4-998d-3565fd698179 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:46.629166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1910-0a46-4cb4-998d-3565fd698179 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:46.629455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1910-0a46-4cb4-998d-3565fd698179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:46.630396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1910-0a46-4cb4-998d-3565fd698179 None admin] RBAC: Adding query filter params (role.id=b3c5094c32f446d88c32145d8b8ae3cd, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=1c8bdac27bde4a2785d4aaad8213baca) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:46.630692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1910-0a46-4cb4-998d-3565fd698179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:46.632118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1910-0a46-4cb4-998d-3565fd698179 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:46.653919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 613/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:28:46 2022] GET /identity/v3/role_assignments?user.id=1c8bdac27bde4a2785d4aaad8213baca&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=b3c5094c32f446d88c32145d8b8ae3cd => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:48.910763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b823d-117c-41ed-a955-0c0ecc27170e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:48.911271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b823d-117c-41ed-a955-0c0ecc27170e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:48.911271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b823d-117c-41ed-a955-0c0ecc27170e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:48.912855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 634/1247] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:28:48.918463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a36d5-93d9-48d8-a6ea-6f50f2584f06 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:48.918904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a36d5-93d9-48d8-a6ea-6f50f2584f06 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:48.919016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a36d5-93d9-48d8-a6ea-6f50f2584f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:48.924373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:48.937095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-362a36d5-93d9-48d8-a6ea-6f50f2584f06 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:28:48.938460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-362a36d5-93d9-48d8-a6ea-6f50f2584f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:28:49.014272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 614/1248] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:28:49.038370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b823d-117c-41ed-a955-0c0ecc27170e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:49.065961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:49.163113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b823d-117c-41ed-a955-0c0ecc27170e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:49.199356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:49.200167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:49.200436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:49.200707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:49.210957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:28:49.219123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:49.220283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:49.221277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 635/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:28:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:49.226823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a36d5-93d9-48d8-a6ea-6f50f2584f06 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:49.346023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a36d5-93d9-48d8-a6ea-6f50f2584f06 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:49.380858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:49.387879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:49.388557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:49.388843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:49.389119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:49.389978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:49.390209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:49.391584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:49.401269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 615/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:49.407446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:49.495604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:49.519431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f663f83-5f0a-479c-b85e-722d57d9d8e5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:49.557022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:49.557913 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:49.558194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:49.558466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:49.569526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:49.571006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:49.588177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:49.589350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:49.590599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:49.590599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:49.590599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:49.590599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:49.594257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 636/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:49.595762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:49.702882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4868c9d-7d9f-4858-870f-817bf25cdb84 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:49.738564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:49.739244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:49.739514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:49.739785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:49.740503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:49.740838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:49.742201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:49.754372 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 616/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:49.760568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:49.872339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eb0ab-e30f-4f00-b734-a2a7daf75756 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:49.900074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:49.909547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:49.910381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:49.910666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:49.910934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:49.918896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:49.920355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:28:49.934510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:49.935661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:49.936834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 637/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:49.943449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:50.005403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:50.055771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fb441-a42a-4c06-bca8-2cabbce17e3b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:50.092323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:50.093015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:50.093298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:50.093571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:50.094463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:50.094774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:50.096268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:50.120057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 617/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Jun 17 11:28:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:50.121331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:50.228772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99eb3f3-794e-4aff-8d39-0e2456bc42cb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:50.263197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:50.263783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:50.264066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:50.264342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:50.323894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:50.337897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:50.339380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:50.450590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 638/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:28:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:28:50.456221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:50.558034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8931d-d0a6-4651-9426-33eb8b370ac8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:50.572097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:50.596548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:50.597344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:50.597758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:50.597936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:50.608119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:50.609544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:50.627550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:50.628667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:50.629969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:50.629969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:50.629969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:50.629969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:50.629969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 618/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 17 11:28:50.635517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:50.727566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:50.739548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa520f0-ade1-4f96-8aa1-a6312765b63e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:50.772752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:50.773494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:50.773782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:50.774250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:50.774977 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:50.775288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:50.776649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:50.788531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 639/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:50.794151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:50.889914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a68f5-03b2-4bea-9102-249bde0b2cc6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:50.922415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:50.923196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:50.923500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:50.923778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:50.930172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:50.931589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:28:50.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:50.946775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:50.947906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 619/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:50.952398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:51.052339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e8096-b7de-43db-bc21-c5ec7a38962a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:51.085418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113cb9a3-9d75-409d-a1e0-08817fc0de22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:51.086105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb9a3-9d75-409d-a1e0-08817fc0de22 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:51.086392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb9a3-9d75-409d-a1e0-08817fc0de22 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:51.086669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb9a3-9d75-409d-a1e0-08817fc0de22 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:51.087459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cb9a3-9d75-409d-a1e0-08817fc0de22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:51.087772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cb9a3-9d75-409d-a1e0-08817fc0de22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:51.089185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cb9a3-9d75-409d-a1e0-08817fc0de22 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:51.100864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 640/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:51.106997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:51.132284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:51.208576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d9ba9-2661-4dfb-9e2d-440c60da037f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:51.241364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691a1e4-da66-4623-a918-2fae304d54e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:51.242045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691a1e4-da66-4623-a918-2fae304d54e1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:51.242331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691a1e4-da66-4623-a918-2fae304d54e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:51.242607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691a1e4-da66-4623-a918-2fae304d54e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:51.243431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691a1e4-da66-4623-a918-2fae304d54e1 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=1c8bdac27bde4a2785d4aaad8213baca) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:51.243744 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691a1e4-da66-4623-a918-2fae304d54e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:51.245162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691a1e4-da66-4623-a918-2fae304d54e1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:51.250499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 620/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:28:51 2022] GET /identity/v3/role_assignments?user.id=1c8bdac27bde4a2785d4aaad8213baca&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:53.350060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c741be-133d-4553-8691-94603776afa5 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:53.350586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c741be-133d-4553-8691-94603776afa5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:53.350682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c741be-133d-4553-8691-94603776afa5 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:53.351998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 641/1261] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:28:53.357130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875861d-00a0-4991-b12f-d755225df138 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:53.357549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875861d-00a0-4991-b12f-d755225df138 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:53.357682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875861d-00a0-4991-b12f-d755225df138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:53.375296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4875861d-00a0-4991-b12f-d755225df138 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:28:53.376628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4875861d-00a0-4991-b12f-d755225df138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:28:53.445950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 621/1262] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:28:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:28:53.468360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c741be-133d-4553-8691-94603776afa5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:53.479632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:53.576935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c741be-133d-4553-8691-94603776afa5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:53.610806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:53.611537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:53.611823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:53.612100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:53.622032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:28:53.629637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:53.630588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:53.631424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 642/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:28:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:53.637011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875861d-00a0-4991-b12f-d755225df138 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:53.655261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:53.732810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875861d-00a0-4991-b12f-d755225df138 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:53.764967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:53.765626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:53.765933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:53.766209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:53.767480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:53.767785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:53.769147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:53.776914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 622/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:28:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:53.783500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:53.848864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:53.882415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d5dfc-40d8-440d-a7a4-80758a05a9f9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:53.915441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:53.916232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:53.916517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:53.916795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:53.927174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:53.928606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:53.945832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:53.946928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:53.948100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:53.948100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:53.948100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:53.948100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:53.948100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 643/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:53.952762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:54.031510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:54.050284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbaecd-8b1c-47ee-b009-f88b6a28bdda None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:54.083260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:54.083922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:54.084209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:54.084488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:54.085209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:54.085521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:54.086926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:54.099143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 623/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:54.104671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:54.204047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7c7d5-3035-4645-b811-f59c2f23c31b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:54.223346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:54.237376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:54.238206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:54.238492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:54.238764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:54.246899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:54.248330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:28:54.262764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:54.263963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:54.265382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 644/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:54.270153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:54.367156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9435b-bad8-45ea-b31e-5c906f84d83a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:54.412519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:54.413188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:54.413467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:54.413737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:54.414602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:54.414907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:54.416317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:54.425272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:54.434017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 624/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Jun 17 11:28:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:28:54.440242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:54.537291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf8162-067c-4968-afce-b6e555c36f90 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:54.571064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:54.571838 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:54.572123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:54.572399 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:54.583289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:54.584723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:54.601832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:54.602879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:54.603989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:54.603989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:54.603989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:54.603989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:54.603989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 645/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:54.608655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:54.705157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b466f-9afc-468a-954a-f3e980756243 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:54.737430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:54.738112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:54.738396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:54.738670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:54.739383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:54.739688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:54.741038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:54.752395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 625/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:54.758296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:54.789691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:54.855304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb534517-632c-405d-8562-09a04f59ecaa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:54.887580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:54.888363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:54.888648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:54.888927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:54.896020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:54.897438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:28:54.912760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:54.916370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:54.918641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 646/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:28:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:54.921077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:54.991126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:55.050156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20ef12-00b8-460c-8b05-8fc1f40f8144 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:55.088169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cf205-a672-4da2-abc9-1d37b4f02bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:55.088833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf205-a672-4da2-abc9-1d37b4f02bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:55.089119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf205-a672-4da2-abc9-1d37b4f02bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:55.089396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf205-a672-4da2-abc9-1d37b4f02bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:55.090212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cf205-a672-4da2-abc9-1d37b4f02bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:55.090524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cf205-a672-4da2-abc9-1d37b4f02bc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:55.091963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cf205-a672-4da2-abc9-1d37b4f02bc9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:55.103738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 626/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:28:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:55.109859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:55.200084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:55.209119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8165-a74f-4829-8557-3a2802fff742 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:55.242479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d009a59-7ccf-4f1e-b5a3-17a6de810560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:55.243363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009a59-7ccf-4f1e-b5a3-17a6de810560 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:55.243645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009a59-7ccf-4f1e-b5a3-17a6de810560 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:55.243915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009a59-7ccf-4f1e-b5a3-17a6de810560 None admin] PATH_INFO: `/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/1c8bdac27bde4a2785d4aaad8213baca/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:55.245867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d009a59-7ccf-4f1e-b5a3-17a6de810560 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc033695b4a4a09a596920a2b36c873, user_id=1c8bdac27bde4a2785d4aaad8213baca, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:55.247283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d009a59-7ccf-4f1e-b5a3-17a6de810560 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:55.260750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 647/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Fri Jun 17 11:28:55 2022] PUT /identity/v3/projects/8fc033695b4a4a09a596920a2b36c873/users/1c8bdac27bde4a2785d4aaad8213baca/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:28:57.337373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9493b-692c-4816-8edd-60c5b49c9f54 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:57.337945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9493b-692c-4816-8edd-60c5b49c9f54 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:57.337945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9493b-692c-4816-8edd-60c5b49c9f54 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:57.339350 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 627/1274] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:28:57.344290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce6e4-5d50-4677-8d4a-dd8c0945f18f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:57.344698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce6e4-5d50-4677-8d4a-dd8c0945f18f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:57.345410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dce6e4-5d50-4677-8d4a-dd8c0945f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:57.361738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-30dce6e4-5d50-4677-8d4a-dd8c0945f18f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:28:57.362984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-30dce6e4-5d50-4677-8d4a-dd8c0945f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:28:57.432287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 648/1275] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:28:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:28:57.456047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9493b-692c-4816-8edd-60c5b49c9f54 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:57.554586 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9493b-692c-4816-8edd-60c5b49c9f54 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:57.571186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:57.588205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:57.588933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:57.589219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:57.589493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:57.599983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 17 11:28:57.607972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:57.609110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:57.610462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 628/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Fri Jun 17 11:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:57.615189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce6e4-5d50-4677-8d4a-dd8c0945f18f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:57.694146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:57.712991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dce6e4-5d50-4677-8d4a-dd8c0945f18f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:57.745726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:57.746413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:57.746695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:57.746973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:57.747712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:57.748018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:57.749364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:57.757655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 649/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Fri Jun 17 11:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:57.763408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:57.861239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7718-493c-47d0-b851-b78fce7d10c3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:57.894050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:57.894834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:57.895114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:57.895390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:57.905548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:57.907289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:57.923815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:57.924860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:57.926020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:57.926020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:57.926020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:57.926020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:57.926020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 629/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:57.930618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:58.026801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a8657-89e2-4a7a-81a6-f2bd67da0d73 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:58.060254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:58.060968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:58.061206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:58.061483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:58.062253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:58.062566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:58.064585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:58.076603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 650/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:58.082305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:58.112712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:58.179619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ac9e0-d711-416f-bf03-1b88fd914f61 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:58.212001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:58.212791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:58.213073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:58.213346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:58.220558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:58.222024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 17 11:28:58.236508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:58.237618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:28:58.238987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 630/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:58.243362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:58.247247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:58.340983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8276d4-cb4c-41e6-931c-e0f76e5a39eb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:58.375296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:58.375891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:58.376791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:58.377067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:58.377948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:58.378262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:58.379674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:58.399506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 651/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Fri Jun 17 11:28:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:58.405212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:58.493879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:58.503615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07647068-dcdc-45fa-b2b1-7ef4a32f32c5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:58.536249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:58.536894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:58.537179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:58.537457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:58.602355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:58.603792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:58.690287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:58.700439 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 631/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Fri Jun 17 11:28:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 17 11:28:58.706843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:58.765793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:58.803918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158555-54ba-4878-9d0c-8c07412d980f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:58.836564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:58.837352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:58.837634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:58.837936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:58.847670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:58.849104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 17 11:28:58.866511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:58.867539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:58.868507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 17 11:28:58.868507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 17 11:28:58.868507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 17 11:28:58.868507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:58.868507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 652/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:58.873243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:58.968099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038277e-2a79-4a3e-96a1-a2e9e260487a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:59.000765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:59.001999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:59.002280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:59.002549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:59.003262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:59.003566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:59.004947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:59.015777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 632/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Fri Jun 17 11:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:59.021215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:59.120910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40124b85-a77c-4046-a9da-75c8a5d13f9b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:59.135054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:59.154922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:59.155711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:59.155992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:59.156264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:59.163607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:59.165027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 17 11:28:59.178941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:28:59.180000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 17 11:28:59.181128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 653/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Fri Jun 17 11:28:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:28:59.185955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:59.235216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:28:59.284954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856a86d-435e-4a39-a677-712d2412fd01 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:59.317413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218087f5-bd9e-4df6-8d3f-81649905c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:59.318100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218087f5-bd9e-4df6-8d3f-81649905c14a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:59.318386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218087f5-bd9e-4df6-8d3f-81649905c14a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:59.318658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218087f5-bd9e-4df6-8d3f-81649905c14a None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:59.319443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218087f5-bd9e-4df6-8d3f-81649905c14a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:59.319753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218087f5-bd9e-4df6-8d3f-81649905c14a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:59.321184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218087f5-bd9e-4df6-8d3f-81649905c14a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:59.332788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 633/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Fri Jun 17 11:28:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:28:59.338955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:28:59.438429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a147d-3139-4b49-97e3-280b7e1a8d7e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:28:59.471041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821adc83-5a2c-46d1-ac61-d6a68d678d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:28:59.471737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821adc83-5a2c-46d1-ac61-d6a68d678d95 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:28:59.472020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821adc83-5a2c-46d1-ac61-d6a68d678d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:28:59.472292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821adc83-5a2c-46d1-ac61-d6a68d678d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:28:59.473135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821adc83-5a2c-46d1-ac61-d6a68d678d95 None admin] RBAC: Adding query filter params (role.id=a0f5f42c562a49678c833a3a1d6e816b, scope.project.id=8fc033695b4a4a09a596920a2b36c873, user.id=1c8bdac27bde4a2785d4aaad8213baca) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:28:59.473451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821adc83-5a2c-46d1-ac61-d6a68d678d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:28:59.474904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821adc83-5a2c-46d1-ac61-d6a68d678d95 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:28:59.480776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 654/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Fri Jun 17 11:28:59 2022] GET /identity/v3/role_assignments?user.id=1c8bdac27bde4a2785d4aaad8213baca&scope.project.id=8fc033695b4a4a09a596920a2b36c873&role.id=a0f5f42c562a49678c833a3a1d6e816b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:29:01.612922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8cfe9-9fc3-49a2-bf32-e6a84595392d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:01.613475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8cfe9-9fc3-49a2-bf32-e6a84595392d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:01.613475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8cfe9-9fc3-49a2-bf32-e6a84595392d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:01.614900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 634/1288] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:29:01.620180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc52a-299b-4d27-b3d8-d98a70fec07d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:01.620620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc52a-299b-4d27-b3d8-d98a70fec07d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:01.620741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc52a-299b-4d27-b3d8-d98a70fec07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:01.638377 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-636cc52a-299b-4d27-b3d8-d98a70fec07d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:29:01.639565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-636cc52a-299b-4d27-b3d8-d98a70fec07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:29:01.668841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:01.705707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 655/1289] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:29:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:29:01.727957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8cfe9-9fc3-49a2-bf32-e6a84595392d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:01.827391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8cfe9-9fc3-49a2-bf32-e6a84595392d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:01.837672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:01.860064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:01.861378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:01.861665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:01.861968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:01.862619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:01.864034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:29:01.875613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:29:01.877427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:01.878657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 635/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:29:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:29:01.883434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cc52a-299b-4d27-b3d8-d98a70fec07d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:01.979525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cc52a-299b-4d27-b3d8-d98a70fec07d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:02.013011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81d652-9083-491c-82d2-8776fe151c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:02.013666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81d652-9083-491c-82d2-8776fe151c06 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:02.013989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81d652-9083-491c-82d2-8776fe151c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:02.014266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81d652-9083-491c-82d2-8776fe151c06 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:02.014967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81d652-9083-491c-82d2-8776fe151c06 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:29:02.015277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81d652-9083-491c-82d2-8776fe151c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:02.016646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81d652-9083-491c-82d2-8776fe151c06 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:02.021560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 656/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Jun 17 11:29:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:29:02.027174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:02.123663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683eb906-db6d-4fb2-8d82-603898063958 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:02.156004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304efd4-e430-4aa1-b53c-f794f84b78c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:02.156655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304efd4-e430-4aa1-b53c-f794f84b78c8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:02.157696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304efd4-e430-4aa1-b53c-f794f84b78c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:02.158006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304efd4-e430-4aa1-b53c-f794f84b78c8 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:02.158706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304efd4-e430-4aa1-b53c-f794f84b78c8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:29:02.159016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304efd4-e430-4aa1-b53c-f794f84b78c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:02.160383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304efd4-e430-4aa1-b53c-f794f84b78c8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:02.164731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 636/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Jun 17 11:29:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:29:04.274174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5b8b4-00dc-44b5-b33f-bdcc11a29dde None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:04.274731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5b8b4-00dc-44b5-b33f-bdcc11a29dde None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:04.274731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5b8b4-00dc-44b5-b33f-bdcc11a29dde None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:04.276102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 657/1293] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:29:04.281247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344873-99ef-43e5-873f-c03488e126d7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:04.281611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344873-99ef-43e5-873f-c03488e126d7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:04.282102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c344873-99ef-43e5-873f-c03488e126d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:04.299162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8c344873-99ef-43e5-873f-c03488e126d7 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:29:04.300346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c344873-99ef-43e5-873f-c03488e126d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:29:04.329298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:04.366120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 637/1294] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:29:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:29:04.389599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5b8b4-00dc-44b5-b33f-bdcc11a29dde None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:04.425700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:04.499299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5b8b4-00dc-44b5-b33f-bdcc11a29dde None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:04.531515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f33fb-3e80-4d47-9519-4ce8ab0233ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:04.532893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f33fb-3e80-4d47-9519-4ce8ab0233ab None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:04.533202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f33fb-3e80-4d47-9519-4ce8ab0233ab None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:04.533556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f33fb-3e80-4d47-9519-4ce8ab0233ab None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:04.534330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f33fb-3e80-4d47-9519-4ce8ab0233ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:04.535695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f33fb-3e80-4d47-9519-4ce8ab0233ab None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:04.548968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 658/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Fri Jun 17 11:29:04 2022] POST /identity/v3/services => generated 246 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:29:06.635647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ae95-8dde-4970-a74a-c2190ef89eb2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:06.636034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ae95-8dde-4970-a74a-c2190ef89eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:06.636197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ae95-8dde-4970-a74a-c2190ef89eb2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:06.637578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 638/1296] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:29:06.643182 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfd3bd-f151-4461-a7c3-68f0c5f10b86 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:06.643595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfd3bd-f151-4461-a7c3-68f0c5f10b86 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:06.643742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfd3bd-f151-4461-a7c3-68f0c5f10b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:06.660754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-62cfd3bd-f151-4461-a7c3-68f0c5f10b86 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:29:06.662031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-62cfd3bd-f151-4461-a7c3-68f0c5f10b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:29:06.733207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 659/1297] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:29:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 17 11:29:06.756202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ae95-8dde-4970-a74a-c2190ef89eb2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:06.853156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ae95-8dde-4970-a74a-c2190ef89eb2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:06.885281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:06.886030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:06.886310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:06.886588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:06.887228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:06.888585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:29:06.900580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:29:06.901732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:06.902814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 639/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:29:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:29:06.907769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfd3bd-f151-4461-a7c3-68f0c5f10b86 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:06.925605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:07.005462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfd3bd-f151-4461-a7c3-68f0c5f10b86 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:07.038219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:07.038934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:07.039346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:07.039624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:07.040327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:29:07.040633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:07.042012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:07.047091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 660/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Jun 17 11:29:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:29:07.054012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:07.084712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:07.151255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1822887-3907-45e9-a1ad-54dcf5462b9f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:07.183243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c9a1-7a2c-4556-839a-1c508033e774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:07.183898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c9a1-7a2c-4556-839a-1c508033e774 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:07.184173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c9a1-7a2c-4556-839a-1c508033e774 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:07.184445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c9a1-7a2c-4556-839a-1c508033e774 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:07.185225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c9a1-7a2c-4556-839a-1c508033e774 None admin] RBAC: Adding query filter params (interface=public, service_id=b5310b2753544a2b979d6f940d3ac870, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:29:07.185532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c9a1-7a2c-4556-839a-1c508033e774 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:07.186920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c9a1-7a2c-4556-839a-1c508033e774 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:07.192319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 640/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Fri Jun 17 11:29:07 2022] GET /identity/v3/endpoints?service_id=b5310b2753544a2b979d6f940d3ac870&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:29:07.198026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:07.281081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:07.300320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5ac5b-494f-4c80-98a6-3821b9389287 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:07.333503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309b90f-fad5-4d76-80a8-3245db48611d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:07.334195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309b90f-fad5-4d76-80a8-3245db48611d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:07.334484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309b90f-fad5-4d76-80a8-3245db48611d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:07.334762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309b90f-fad5-4d76-80a8-3245db48611d None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:07.335443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309b90f-fad5-4d76-80a8-3245db48611d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:07.336805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309b90f-fad5-4d76-80a8-3245db48611d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:07.343528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 661/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Fri Jun 17 11:29:07 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:29:09.456734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c8ba-a6c6-4be5-b87a-3e059a9c40d0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:09.457260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c8ba-a6c6-4be5-b87a-3e059a9c40d0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:09.457260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c8ba-a6c6-4be5-b87a-3e059a9c40d0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:09.458678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 641/1302] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:29:09.463828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92fbbe-f124-465f-a828-d07fb97897e3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:09.464234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92fbbe-f124-465f-a828-d07fb97897e3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:09.464393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92fbbe-f124-465f-a828-d07fb97897e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:09.481275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9e92fbbe-f124-465f-a828-d07fb97897e3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:29:09.483139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9e92fbbe-f124-465f-a828-d07fb97897e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:29:09.550265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 662/1303] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:29:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 17 11:29:09.573813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c8ba-a6c6-4be5-b87a-3e059a9c40d0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:09.667142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:09.675244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c8ba-a6c6-4be5-b87a-3e059a9c40d0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:09.707292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:09.708613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:09.708900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:09.709175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:09.709821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:09.711205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 17 11:29:09.723379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 17 11:29:09.724553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 17 11:29:09.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 642/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Fri Jun 17 11:29:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 17 11:29:09.730538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92fbbe-f124-465f-a828-d07fb97897e3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:09.804210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:09.826365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92fbbe-f124-465f-a828-d07fb97897e3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:09.859018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bd559-0d02-4ca3-a2ce-145d6920f654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:09.859668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd559-0d02-4ca3-a2ce-145d6920f654 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:09.859953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd559-0d02-4ca3-a2ce-145d6920f654 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:09.860229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bd559-0d02-4ca3-a2ce-145d6920f654 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:09.860932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bd559-0d02-4ca3-a2ce-145d6920f654 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:29:09.861784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bd559-0d02-4ca3-a2ce-145d6920f654 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:09.863168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bd559-0d02-4ca3-a2ce-145d6920f654 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:09.868257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 663/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Fri Jun 17 11:29:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:29:09.873870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:29:09.970364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dfd4a-62e9-4328-adfc-691b84a6441d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:29:10.003122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20eadf-c25f-40ef-bf51-be23238beb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:29:10.003765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20eadf-c25f-40ef-bf51-be23238beb0c None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:29:10.004049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20eadf-c25f-40ef-bf51-be23238beb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:29:10.004393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20eadf-c25f-40ef-bf51-be23238beb0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:29:10.005144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20eadf-c25f-40ef-bf51-be23238beb0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:29:10.006530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20eadf-c25f-40ef-bf51-be23238beb0c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:29:10.024418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 643/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Fri Jun 17 11:29:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:29:19.865260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=30 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:29:20.020159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:12.885381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48d46d-bec2-4085-8ca2-b043570aae93 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:12.885948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48d46d-bec2-4085-8ca2-b043570aae93 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:12.885948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48d46d-bec2-4085-8ca2-b043570aae93 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:12.892521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe1d46-dfc3-452f-b2c7-98ad704ad578 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:12.892969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe1d46-dfc3-452f-b2c7-98ad704ad578 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:12.893066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe1d46-dfc3-452f-b2c7-98ad704ad578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:12.896586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 664/1307] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:12.971243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-81fe1d46-dfc3-452f-b2c7-98ad704ad578 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:12.972685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-81fe1d46-dfc3-452f-b2c7-98ad704ad578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:13.126137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 644/1308] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:13.178140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4482d-57f4-455f-b5b1-e175a952aca2 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:13.178564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4482d-57f4-455f-b5b1-e175a952aca2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:13.178903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4482d-57f4-455f-b5b1-e175a952aca2 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:13.183334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 665/1309] 10.222.0.25 () {54 vars in 964 bytes} [Fri Jun 17 11:30:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:13.192309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81cecc-78cb-4f90-9cb9-bda186d44df4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:13.192627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81cecc-78cb-4f90-9cb9-bda186d44df4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:13.192933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81cecc-78cb-4f90-9cb9-bda186d44df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:13.340645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5b81cecc-78cb-4f90-9cb9-bda186d44df4 None None] MFA Rules not processed for user `b3b35d6d2cd848f4841352fbb58f2053`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:13.342719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5b81cecc-78cb-4f90-9cb9-bda186d44df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:13.389005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:13.452810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 645/1310] 10.222.0.25 () {58 vars in 1061 bytes} [Fri Jun 17 11:30:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:13.462393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4482d-57f4-455f-b5b1-e175a952aca2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:13.464121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bca4482d-57f4-455f-b5b1-e175a952aca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:13.588072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4482d-57f4-455f-b5b1-e175a952aca2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:13.679484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5390a27-7cc8-4cd7-be56-0cf534e561c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:13.680066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390a27-7cc8-4cd7-be56-0cf534e561c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:13.680346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390a27-7cc8-4cd7-be56-0cf534e561c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:13.680621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5390a27-7cc8-4cd7-be56-0cf534e561c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:13.682374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d5390a27-7cc8-4cd7-be56-0cf534e561c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:13.746139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:13.812826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5390a27-7cc8-4cd7-be56-0cf534e561c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:13.814311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5390a27-7cc8-4cd7-be56-0cf534e561c7 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:13.836073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 666/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:16.307705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c88dd-e8f9-4460-8112-c7fbe2bb9794 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:16.308277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c88dd-e8f9-4460-8112-c7fbe2bb9794 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:16.308277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c88dd-e8f9-4460-8112-c7fbe2bb9794 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:16.309643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 646/1312] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:16.315323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e680e-8e49-4c96-8058-bddb5acf18c8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:16.315738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e680e-8e49-4c96-8058-bddb5acf18c8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:16.315858 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e680e-8e49-4c96-8058-bddb5acf18c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:16.396294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-905e680e-8e49-4c96-8058-bddb5acf18c8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:16.397493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-905e680e-8e49-4c96-8058-bddb5acf18c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:16.494271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 667/1313] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:16.531987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c88dd-e8f9-4460-8112-c7fbe2bb9794 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:16.533516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2a3c88dd-e8f9-4460-8112-c7fbe2bb9794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:16.642101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c88dd-e8f9-4460-8112-c7fbe2bb9794 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:16.720219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:16.723317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de2718-c4a8-463f-bc52-65dc17a5c055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:16.724038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de2718-c4a8-463f-bc52-65dc17a5c055 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:16.724524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de2718-c4a8-463f-bc52-65dc17a5c055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:16.724842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de2718-c4a8-463f-bc52-65dc17a5c055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:16.727072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b2de2718-c4a8-463f-bc52-65dc17a5c055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:16.843374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de2718-c4a8-463f-bc52-65dc17a5c055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:16.844822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de2718-c4a8-463f-bc52-65dc17a5c055 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:16.866421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 647/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:19.309955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a96142-0a32-4275-b039-f3ead2eb1503 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:19.311016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a96142-0a32-4275-b039-f3ead2eb1503 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:19.311113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a96142-0a32-4275-b039-f3ead2eb1503 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:19.312444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 668/1315] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:19.317574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d6f08-5f0d-4e97-b455-9134c8613650 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:19.317994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d6f08-5f0d-4e97-b455-9134c8613650 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:19.318176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d6f08-5f0d-4e97-b455-9134c8613650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:19.383125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-179d6f08-5f0d-4e97-b455-9134c8613650 None None] MFA Rules not processed for user `4466175761d84340a500132eb2b738d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:19.384352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-179d6f08-5f0d-4e97-b455-9134c8613650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:19.436840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:19.481601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 648/1316] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:19 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:19.521287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a96142-0a32-4275-b039-f3ead2eb1503 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:19.522851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d4a96142-0a32-4275-b039-f3ead2eb1503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:19.533412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:19.625865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a96142-0a32-4275-b039-f3ead2eb1503 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:19.699803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241f7bf-9007-4263-af83-216c89b02f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:19.700458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f7bf-9007-4263-af83-216c89b02f3d service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:19.700762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f7bf-9007-4263-af83-216c89b02f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:19.701040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f7bf-9007-4263-af83-216c89b02f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:19.702750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a241f7bf-9007-4263-af83-216c89b02f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:19.765660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:19.812276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241f7bf-9007-4263-af83-216c89b02f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:19.813708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241f7bf-9007-4263-af83-216c89b02f3d service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:19.834267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 669/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:19 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:22.476048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd723cf-8d99-4a32-830c-a9b9c47b4870 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:22.476607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd723cf-8d99-4a32-830c-a9b9c47b4870 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:22.476607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd723cf-8d99-4a32-830c-a9b9c47b4870 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:22.477999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 649/1318] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:22.483225 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf261-95f0-48a6-9c86-46b7a4a1171f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:22.483632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf261-95f0-48a6-9c86-46b7a4a1171f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:22.483788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf261-95f0-48a6-9c86-46b7a4a1171f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:22.547257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-03bbf261-95f0-48a6-9c86-46b7a4a1171f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:22.548460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-03bbf261-95f0-48a6-9c86-46b7a4a1171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:22.643852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 670/1319] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:22.682170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd723cf-8d99-4a32-830c-a9b9c47b4870 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:22.683687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9bd723cf-8d99-4a32-830c-a9b9c47b4870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:22.788466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd723cf-8d99-4a32-830c-a9b9c47b4870 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:22.859935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad1af2-e2c2-416f-bed0-2dec3029aaa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:22.860526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad1af2-e2c2-416f-bed0-2dec3029aaa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:22.860812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad1af2-e2c2-416f-bed0-2dec3029aaa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:22.861093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad1af2-e2c2-416f-bed0-2dec3029aaa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:22.862763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-71ad1af2-e2c2-416f-bed0-2dec3029aaa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:22.870041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:22.968985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad1af2-e2c2-416f-bed0-2dec3029aaa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:22.970468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad1af2-e2c2-416f-bed0-2dec3029aaa3 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:22.988450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 650/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:25.776316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797709e-8c83-4a95-b8a7-1d9335f6c5e5 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:25.776846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797709e-8c83-4a95-b8a7-1d9335f6c5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:25.776846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797709e-8c83-4a95-b8a7-1d9335f6c5e5 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:25.778276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 671/1321] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:25.785187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dd38e-cd62-4245-9c0d-9fad27327dc3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:25.785659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dd38e-cd62-4245-9c0d-9fad27327dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:25.785754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103dd38e-cd62-4245-9c0d-9fad27327dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:25.854327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-103dd38e-cd62-4245-9c0d-9fad27327dc3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:25.855571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-103dd38e-cd62-4245-9c0d-9fad27327dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:25.920558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:25.963429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 651/1322] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:25.999674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797709e-8c83-4a95-b8a7-1d9335f6c5e5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:26.001245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b797709e-8c83-4a95-b8a7-1d9335f6c5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:26.023206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:26.108135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797709e-8c83-4a95-b8a7-1d9335f6c5e5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:26.180560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834764dc-51d0-4ff2-b3f7-38d398e4450c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:26.181143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834764dc-51d0-4ff2-b3f7-38d398e4450c service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:26.181427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834764dc-51d0-4ff2-b3f7-38d398e4450c service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:26.181702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834764dc-51d0-4ff2-b3f7-38d398e4450c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:26.183457 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-834764dc-51d0-4ff2-b3f7-38d398e4450c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:26.254463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:26.293961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834764dc-51d0-4ff2-b3f7-38d398e4450c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:26.295386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834764dc-51d0-4ff2-b3f7-38d398e4450c service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:26.324828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 672/1323] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:28.509459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5dedf-1039-45c9-b5ae-3796b4b2838b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:28.510059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5dedf-1039-45c9-b5ae-3796b4b2838b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:28.510059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5dedf-1039-45c9-b5ae-3796b4b2838b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:28.511424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 652/1324] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:28.517603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ab14-429e-442f-901d-3d0e43d89e2a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:28.518219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ab14-429e-442f-901d-3d0e43d89e2a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:28.518389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ab14-429e-442f-901d-3d0e43d89e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:28.583449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1da9ab14-429e-442f-901d-3d0e43d89e2a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:28.584684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1da9ab14-429e-442f-901d-3d0e43d89e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:28.680567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 673/1325] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:28.732298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74633b28-e72d-4ec6-9e6f-d72d29ebc9a9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:28.732686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74633b28-e72d-4ec6-9e6f-d72d29ebc9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:28.732980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74633b28-e72d-4ec6-9e6f-d72d29ebc9a9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:28.734441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 653/1326] 10.222.0.25 () {54 vars in 964 bytes} [Fri Jun 17 11:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:28.740850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf830-3ac2-44ea-88f4-2f505814aaf9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:28.741228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf830-3ac2-44ea-88f4-2f505814aaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:28.741516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cf830-3ac2-44ea-88f4-2f505814aaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:28.771707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:28.822096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a65cf830-3ac2-44ea-88f4-2f505814aaf9 None None] MFA Rules not processed for user `b3b35d6d2cd848f4841352fbb58f2053`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:28.823335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a65cf830-3ac2-44ea-88f4-2f505814aaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:28.899230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 674/1327] 10.222.0.25 () {58 vars in 1061 bytes} [Fri Jun 17 11:30:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:28.908500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74633b28-e72d-4ec6-9e6f-d72d29ebc9a9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:28.910162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-74633b28-e72d-4ec6-9e6f-d72d29ebc9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:29.018817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74633b28-e72d-4ec6-9e6f-d72d29ebc9a9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:29.093560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81421d-4bbc-4456-a2e8-e63a3a8f4b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:29.094250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81421d-4bbc-4456-a2e8-e63a3a8f4b59 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:29.094540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81421d-4bbc-4456-a2e8-e63a3a8f4b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:29.094814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81421d-4bbc-4456-a2e8-e63a3a8f4b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:29.096463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bd81421d-4bbc-4456-a2e8-e63a3a8f4b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:29.113596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:29.205047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81421d-4bbc-4456-a2e8-e63a3a8f4b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:29.206537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81421d-4bbc-4456-a2e8-e63a3a8f4b59 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:29.224827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 654/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:31.641048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08705f89-2afe-43b1-ba9c-a6898a688240 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:31.641512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08705f89-2afe-43b1-ba9c-a6898a688240 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:31.641607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08705f89-2afe-43b1-ba9c-a6898a688240 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:31.643047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 675/1329] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:31.648218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aa9d3-24d4-46ff-84fb-3e90e5836a57 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:31.648601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aa9d3-24d4-46ff-84fb-3e90e5836a57 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:31.648778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aa9d3-24d4-46ff-84fb-3e90e5836a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:31.712724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5f0aa9d3-24d4-46ff-84fb-3e90e5836a57 None None] MFA Rules not processed for user `4466175761d84340a500132eb2b738d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:31.713953 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5f0aa9d3-24d4-46ff-84fb-3e90e5836a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:31.780171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:31.812506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 655/1330] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:31.850031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08705f89-2afe-43b1-ba9c-a6898a688240 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:31.851572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08705f89-2afe-43b1-ba9c-a6898a688240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:31.959486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08705f89-2afe-43b1-ba9c-a6898a688240 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:31.982146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:32.036165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b566c-5e5e-4a25-9ce3-dbfd2e822080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:32.036822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b566c-5e5e-4a25-9ce3-dbfd2e822080 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:32.037135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b566c-5e5e-4a25-9ce3-dbfd2e822080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:32.037412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b566c-5e5e-4a25-9ce3-dbfd2e822080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:32.039139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5a4b566c-5e5e-4a25-9ce3-dbfd2e822080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:32.151054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b566c-5e5e-4a25-9ce3-dbfd2e822080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:32.152505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b566c-5e5e-4a25-9ce3-dbfd2e822080 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:32.171532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 676/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:34.712589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af208-7250-4ab2-9ef6-41733a1d0899 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:34.713117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af208-7250-4ab2-9ef6-41733a1d0899 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:34.713117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af208-7250-4ab2-9ef6-41733a1d0899 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:34.714544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 656/1332] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:34.720083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03198af-f1f6-4f2c-9991-f8ff9e3bae3b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:34.720486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03198af-f1f6-4f2c-9991-f8ff9e3bae3b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:34.720646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03198af-f1f6-4f2c-9991-f8ff9e3bae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:34.782774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:34.785546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b03198af-f1f6-4f2c-9991-f8ff9e3bae3b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:34.787492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b03198af-f1f6-4f2c-9991-f8ff9e3bae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:34.882668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 677/1333] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:34.919139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367af208-7250-4ab2-9ef6-41733a1d0899 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:34.920745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-367af208-7250-4ab2-9ef6-41733a1d0899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:35.021145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367af208-7250-4ab2-9ef6-41733a1d0899 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:35.095016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0afc8-7fb4-4e76-911a-2e7fc4bc3263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:35.095677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0afc8-7fb4-4e76-911a-2e7fc4bc3263 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:35.095958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0afc8-7fb4-4e76-911a-2e7fc4bc3263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:35.096234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0afc8-7fb4-4e76-911a-2e7fc4bc3263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:35.097904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-70b0afc8-7fb4-4e76-911a-2e7fc4bc3263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:35.125355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:35.212304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0afc8-7fb4-4e76-911a-2e7fc4bc3263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:35.213757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0afc8-7fb4-4e76-911a-2e7fc4bc3263 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:35.232048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 657/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:37.396931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041dc65-07e9-49d0-8ab3-20ddcbcc4bd8 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:37.397454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041dc65-07e9-49d0-8ab3-20ddcbcc4bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:37.397454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041dc65-07e9-49d0-8ab3-20ddcbcc4bd8 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:37.398837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 678/1335] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:37.404830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57002f86-4b7b-407b-b175-64bd513a17a1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:37.405248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57002f86-4b7b-407b-b175-64bd513a17a1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:37.405424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57002f86-4b7b-407b-b175-64bd513a17a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:37.468298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-57002f86-4b7b-407b-b175-64bd513a17a1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:37.469479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-57002f86-4b7b-407b-b175-64bd513a17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:37.542885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:37.565279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 658/1336] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:37.604211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041dc65-07e9-49d0-8ab3-20ddcbcc4bd8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:37.605750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6041dc65-07e9-49d0-8ab3-20ddcbcc4bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:37.707559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041dc65-07e9-49d0-8ab3-20ddcbcc4bd8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:37.736765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:37.782454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa7e73-1f52-42ab-9f91-fdda81d8f46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:37.783111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa7e73-1f52-42ab-9f91-fdda81d8f46f service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:37.783397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa7e73-1f52-42ab-9f91-fdda81d8f46f service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:37.783674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa7e73-1f52-42ab-9f91-fdda81d8f46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:37.785322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5bfa7e73-1f52-42ab-9f91-fdda81d8f46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:37.892437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa7e73-1f52-42ab-9f91-fdda81d8f46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:37.893882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa7e73-1f52-42ab-9f91-fdda81d8f46f service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:37.912612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 679/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:40.513016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211f15-a486-4dff-86d2-bc2fab4d421f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:40.513551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211f15-a486-4dff-86d2-bc2fab4d421f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:40.513664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4211f15-a486-4dff-86d2-bc2fab4d421f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:40.515010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 659/1338] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:40.520148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ddb5-af95-4e3f-894e-6b82411784c7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:40.520525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ddb5-af95-4e3f-894e-6b82411784c7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:40.520701 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73ddb5-af95-4e3f-894e-6b82411784c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:40.584670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ac73ddb5-af95-4e3f-894e-6b82411784c7 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:40.585927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ac73ddb5-af95-4e3f-894e-6b82411784c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:40.594666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:40.686812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 680/1339] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:40.722962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211f15-a486-4dff-86d2-bc2fab4d421f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:40.724468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f4211f15-a486-4dff-86d2-bc2fab4d421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:40.826399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4211f15-a486-4dff-86d2-bc2fab4d421f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:40.897279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3396e0-2118-4089-9ef3-623e1604dd33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:40.897958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3396e0-2118-4089-9ef3-623e1604dd33 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:40.898241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3396e0-2118-4089-9ef3-623e1604dd33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:40.898515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3396e0-2118-4089-9ef3-623e1604dd33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:40.900133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb3396e0-2118-4089-9ef3-623e1604dd33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:40.939277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:41.009977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3396e0-2118-4089-9ef3-623e1604dd33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:41.011410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3396e0-2118-4089-9ef3-623e1604dd33 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:41.030618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 660/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:30:45.126782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f581dc-f98b-4be0-a53e-44b8f7355cb9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:45.127356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f581dc-f98b-4be0-a53e-44b8f7355cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:45.127356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f581dc-f98b-4be0-a53e-44b8f7355cb9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:45.128851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 681/1341] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:45.134021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a83d-8c97-47d6-858d-95209671cb60 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:45.134468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a83d-8c97-47d6-858d-95209671cb60 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:45.134563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a83d-8c97-47d6-858d-95209671cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:45.205991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6953a83d-8c97-47d6-858d-95209671cb60 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:45.207061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6953a83d-8c97-47d6-858d-95209671cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:45.304960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:45.318609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 661/1342] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:30:45.357643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f581dc-f98b-4be0-a53e-44b8f7355cb9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:45.360906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c5f581dc-f98b-4be0-a53e-44b8f7355cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:45.463269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f581dc-f98b-4be0-a53e-44b8f7355cb9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:45.466909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27695-ca4b-4ad9-9190-3d062f92e044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:45.467586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27695-ca4b-4ad9-9190-3d062f92e044 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:45.467852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27695-ca4b-4ad9-9190-3d062f92e044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:45.468130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27695-ca4b-4ad9-9190-3d062f92e044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:45.477646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd27695-ca4b-4ad9-9190-3d062f92e044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:45.479181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd27695-ca4b-4ad9-9190-3d062f92e044 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:45.489896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 682/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:48.231803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e709a44-e95e-4eba-97ab-68cfe94c7b6b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:48.232373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e709a44-e95e-4eba-97ab-68cfe94c7b6b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:48.232549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e709a44-e95e-4eba-97ab-68cfe94c7b6b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:48.234109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 662/1344] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:48.239367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3dd79-98ab-43e2-a67b-ab9a1548b00f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:48.239774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3dd79-98ab-43e2-a67b-ab9a1548b00f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:48.239936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3dd79-98ab-43e2-a67b-ab9a1548b00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:48.268583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-98d3dd79-98ab-43e2-a67b-ab9a1548b00f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:48.269823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-98d3dd79-98ab-43e2-a67b-ab9a1548b00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:48.301791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:30:48.354861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 683/1345] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:48.393708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e709a44-e95e-4eba-97ab-68cfe94c7b6b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:48.407825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e709a44-e95e-4eba-97ab-68cfe94c7b6b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:48.412231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319f180-b1a4-4b5f-b454-0a804b7e00bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:48.412911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f180-b1a4-4b5f-b454-0a804b7e00bc service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:48.413201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f180-b1a4-4b5f-b454-0a804b7e00bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:48.413514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319f180-b1a4-4b5f-b454-0a804b7e00bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:48.423682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319f180-b1a4-4b5f-b454-0a804b7e00bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:48.425173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319f180-b1a4-4b5f-b454-0a804b7e00bc service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:48.435624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 663/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:52.818106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b430c9-b8a4-4cf5-b178-c9d70ce90542 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:52.818644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b430c9-b8a4-4cf5-b178-c9d70ce90542 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:52.818644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b430c9-b8a4-4cf5-b178-c9d70ce90542 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:52.820081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 684/1347] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:52.825381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dec60f-c75a-4a07-a547-265b56f05d6f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:52.825797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dec60f-c75a-4a07-a547-265b56f05d6f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:52.826018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dec60f-c75a-4a07-a547-265b56f05d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:52.843485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a5dec60f-c75a-4a07-a547-265b56f05d6f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:52.844697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5dec60f-c75a-4a07-a547-265b56f05d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:52.852854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 664/1348] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:30:52.890801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b430c9-b8a4-4cf5-b178-c9d70ce90542 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:30:52.892510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e9b430c9-b8a4-4cf5-b178-c9d70ce90542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:52.905465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b430c9-b8a4-4cf5-b178-c9d70ce90542 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:30:52.909211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac6e15-65e5-4e95-a583-2bd95337b82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:30:52.909974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac6e15-65e5-4e95-a583-2bd95337b82d service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:52.910263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac6e15-65e5-4e95-a583-2bd95337b82d service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:52.910544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac6e15-65e5-4e95-a583-2bd95337b82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:52.920404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac6e15-65e5-4e95-a583-2bd95337b82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:30:52.921890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac6e15-65e5-4e95-a583-2bd95337b82d service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:30:52.932118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 685/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:30:58.030112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aaf13-1eac-4a04-8c94-6c23d645a719 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:58.030989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aaf13-1eac-4a04-8c94-6c23d645a719 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:58.031443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414aaf13-1eac-4a04-8c94-6c23d645a719 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:58.033393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 665/1350] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:30:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:30:58.038998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fb208-06e0-477a-9691-a4d7685bf02e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:30:58.039793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fb208-06e0-477a-9691-a4d7685bf02e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:30:58.040224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fb208-06e0-477a-9691-a4d7685bf02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:30:58.166315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0b8fb208-06e0-477a-9691-a4d7685bf02e None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:30:58.167602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0b8fb208-06e0-477a-9691-a4d7685bf02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:30:58.254941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 686/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:30:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:02.570855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e90ac-ad6c-4191-a67d-5cd033fbde51 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:02.571386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e90ac-ad6c-4191-a67d-5cd033fbde51 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:02.571386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e90ac-ad6c-4191-a67d-5cd033fbde51 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:02.572964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 666/1352] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:02.578191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1242c7-73ab-4112-970e-511d3ae4ff6b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:02.579231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1242c7-73ab-4112-970e-511d3ae4ff6b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:02.579739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1242c7-73ab-4112-970e-511d3ae4ff6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:02.598019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bb1242c7-73ab-4112-970e-511d3ae4ff6b None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:02.599224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bb1242c7-73ab-4112-970e-511d3ae4ff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:02.608622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 687/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:31:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:02.838371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=13 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:31:03.999629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746472a-0e0f-4f52-a533-730e908cee7d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:04.000245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746472a-0e0f-4f52-a533-730e908cee7d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:04.000245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746472a-0e0f-4f52-a533-730e908cee7d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:04.001549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 667/1354] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:04.006618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632074f-5c57-4d5c-984e-7c765d50eb0c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:04.007062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632074f-5c57-4d5c-984e-7c765d50eb0c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:04.007159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632074f-5c57-4d5c-984e-7c765d50eb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:04.027306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f632074f-5c57-4d5c-984e-7c765d50eb0c None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:04.028771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f632074f-5c57-4d5c-984e-7c765d50eb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:04.037521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 688/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:31:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:07.824983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4aad-a65f-4595-a69b-55d74c2919a8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:07.825828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4aad-a65f-4595-a69b-55d74c2919a8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:07.826282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4aad-a65f-4595-a69b-55d74c2919a8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:07.827805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 668/1356] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:07.833290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62381c8-0c9d-42c2-a14b-71e059daf9b5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:07.833738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62381c8-0c9d-42c2-a14b-71e059daf9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:07.834319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62381c8-0c9d-42c2-a14b-71e059daf9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:07.854013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b62381c8-0c9d-42c2-a14b-71e059daf9b5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:07.855427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b62381c8-0c9d-42c2-a14b-71e059daf9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:07.863541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 689/1357] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:07.919227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a33daa-3285-4638-a6dd-03fd99e6ea15 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:07.919735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a33daa-3285-4638-a6dd-03fd99e6ea15 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:07.920204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a33daa-3285-4638-a6dd-03fd99e6ea15 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:07.922455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 669/1358] 10.222.0.25 () {54 vars in 961 bytes} [Fri Jun 17 11:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:07.931939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934f0c6-9b61-4c08-bb20-a66b597331cb None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:07.935595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934f0c6-9b61-4c08-bb20-a66b597331cb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:07.936167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934f0c6-9b61-4c08-bb20-a66b597331cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:07.982066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:31:07.991411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5934f0c6-9b61-4c08-bb20-a66b597331cb None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:07.992916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5934f0c6-9b61-4c08-bb20-a66b597331cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:08.084153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 690/1359] 10.222.0.25 () {58 vars in 1058 bytes} [Fri Jun 17 11:31:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:31:08.092759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a33daa-3285-4638-a6dd-03fd99e6ea15 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:08.107274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a33daa-3285-4638-a6dd-03fd99e6ea15 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:08.112835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730733e5-1cab-4173-a09f-48913340d182 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:08.113647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730733e5-1cab-4173-a09f-48913340d182 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:08.114021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730733e5-1cab-4173-a09f-48913340d182 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:08.114394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730733e5-1cab-4173-a09f-48913340d182 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:08.123633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730733e5-1cab-4173-a09f-48913340d182 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:08.125323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730733e5-1cab-4173-a09f-48913340d182 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:08.137816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 670/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:10.476822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89f723-df70-4d79-86fb-f2674dd74f07 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:10.477759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89f723-df70-4d79-86fb-f2674dd74f07 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:10.478138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89f723-df70-4d79-86fb-f2674dd74f07 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:10.480120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 691/1361] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:10.486160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63b8c6-9a11-47c2-ba56-5337d8711a70 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:10.486655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63b8c6-9a11-47c2-ba56-5337d8711a70 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:10.487169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63b8c6-9a11-47c2-ba56-5337d8711a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:10.506060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ec63b8c6-9a11-47c2-ba56-5337d8711a70 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:10.507296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ec63b8c6-9a11-47c2-ba56-5337d8711a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:10.515730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 671/1362] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:10.562546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89f723-df70-4d79-86fb-f2674dd74f07 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:10.578281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89f723-df70-4d79-86fb-f2674dd74f07 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:10.584526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e1070-99ce-4d7b-ab7c-0c848a2a8027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:10.585507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1070-99ce-4d7b-ab7c-0c848a2a8027 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:10.586029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1070-99ce-4d7b-ab7c-0c848a2a8027 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:10.586494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1070-99ce-4d7b-ab7c-0c848a2a8027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:10.596778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e1070-99ce-4d7b-ab7c-0c848a2a8027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:10.598947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e1070-99ce-4d7b-ab7c-0c848a2a8027 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:10.610702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 692/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:13.072522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a3db2-fde6-4e2b-8761-8c3504fb8ced None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:13.073047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a3db2-fde6-4e2b-8761-8c3504fb8ced None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:13.073047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a3db2-fde6-4e2b-8761-8c3504fb8ced None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:13.074515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 672/1364] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:31:13.079711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79badd8b-6361-4c2f-9a5c-8f160952f2ba None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:13.080115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79badd8b-6361-4c2f-9a5c-8f160952f2ba None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:13.080272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79badd8b-6361-4c2f-9a5c-8f160952f2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:13.098198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-79badd8b-6361-4c2f-9a5c-8f160952f2ba None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:13.099402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-79badd8b-6361-4c2f-9a5c-8f160952f2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:13.107894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 693/1365] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:31:13.155003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27794-691e-4c83-9f5f-e54a3634e2ec None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:13.155288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27794-691e-4c83-9f5f-e54a3634e2ec None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:13.155569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27794-691e-4c83-9f5f-e54a3634e2ec None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:13.156955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 673/1366] 10.222.0.25 () {54 vars in 961 bytes} [Fri Jun 17 11:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:13.162514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944095-f417-4388-bacf-575837752b71 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:13.162924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944095-f417-4388-bacf-575837752b71 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:13.163502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e944095-f417-4388-bacf-575837752b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:13.183246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3e944095-f417-4388-bacf-575837752b71 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:13.184428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3e944095-f417-4388-bacf-575837752b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:13.193460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 694/1367] 10.222.0.25 () {58 vars in 1058 bytes} [Fri Jun 17 11:31:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:13.201098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed27794-691e-4c83-9f5f-e54a3634e2ec None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:13.215364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed27794-691e-4c83-9f5f-e54a3634e2ec None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:13.221188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7402316-fc5e-46ad-a35b-5bd812cec8e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:13.221833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402316-fc5e-46ad-a35b-5bd812cec8e2 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:13.222166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402316-fc5e-46ad-a35b-5bd812cec8e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:13.222444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7402316-fc5e-46ad-a35b-5bd812cec8e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:13.231604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7402316-fc5e-46ad-a35b-5bd812cec8e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:13.233191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7402316-fc5e-46ad-a35b-5bd812cec8e2 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:13.249075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 674/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:15.489082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd484-197d-4918-bb7e-23b4586353c4 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:15.489674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd484-197d-4918-bb7e-23b4586353c4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:15.489674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd484-197d-4918-bb7e-23b4586353c4 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:15.491052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 695/1369] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:15.496365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f11f0-4c1b-4518-8e0b-0711a019dfe6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:15.497185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f11f0-4c1b-4518-8e0b-0711a019dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:15.497672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f11f0-4c1b-4518-8e0b-0711a019dfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:15.516263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-029f11f0-4c1b-4518-8e0b-0711a019dfe6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:15.517721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-029f11f0-4c1b-4518-8e0b-0711a019dfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:15.526380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 675/1370] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:15.570161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dd484-197d-4918-bb7e-23b4586353c4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:15.585339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dd484-197d-4918-bb7e-23b4586353c4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:15.591722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d455930-0a90-4147-bf9f-1a77283e4bcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:15.592607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d455930-0a90-4147-bf9f-1a77283e4bcb service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:15.593078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d455930-0a90-4147-bf9f-1a77283e4bcb service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:15.593550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d455930-0a90-4147-bf9f-1a77283e4bcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:15.602584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d455930-0a90-4147-bf9f-1a77283e4bcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:15.604430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d455930-0a90-4147-bf9f-1a77283e4bcb service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:15.615899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 696/1371] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:16.039892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919d8a2-964a-451c-b166-a7af6b6540f8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:16.040184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919d8a2-964a-451c-b166-a7af6b6540f8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:16.040650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919d8a2-964a-451c-b166-a7af6b6540f8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:16.042620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 676/1372] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:16.047941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e07e0-2ed9-42c6-b0e8-bd81badaf5a4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:16.048241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e07e0-2ed9-42c6-b0e8-bd81badaf5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:16.048527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e07e0-2ed9-42c6-b0e8-bd81badaf5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:16.067122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3f6e07e0-2ed9-42c6-b0e8-bd81badaf5a4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:16.068331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3f6e07e0-2ed9-42c6-b0e8-bd81badaf5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:16.076652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 697/1373] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:16.386257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641144e-a92b-4e05-85dd-65a86c7a40f9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:16.386257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641144e-a92b-4e05-85dd-65a86c7a40f9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:16.386257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641144e-a92b-4e05-85dd-65a86c7a40f9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:16.388711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 677/1374] 10.222.0.25 () {54 vars in 963 bytes} [Fri Jun 17 11:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:31:16.392994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf56b0-cf82-4d15-99b2-6172d7ca5d9a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:16.392994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf56b0-cf82-4d15-99b2-6172d7ca5d9a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:16.392994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf56b0-cf82-4d15-99b2-6172d7ca5d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:16.444031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dfaf56b0-cf82-4d15-99b2-6172d7ca5d9a None None] MFA Rules not processed for user `17154b0972cb4534a5c379babcb9d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:16.445236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dfaf56b0-cf82-4d15-99b2-6172d7ca5d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:16.510457 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 698/1375] 10.222.0.25 () {58 vars in 1060 bytes} [Fri Jun 17 11:31:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:16.517805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641144e-a92b-4e05-85dd-65a86c7a40f9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:16.529319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641144e-a92b-4e05-85dd-65a86c7a40f9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:16.532848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aafe9c-ba4f-4502-8a6e-65163ca15ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:16.533492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aafe9c-ba4f-4502-8a6e-65163ca15ffd service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:16.533775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aafe9c-ba4f-4502-8a6e-65163ca15ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:16.534088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aafe9c-ba4f-4502-8a6e-65163ca15ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:16.543787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aafe9c-ba4f-4502-8a6e-65163ca15ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:16.545275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aafe9c-ba4f-4502-8a6e-65163ca15ffd service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:16.561371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 678/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:17.795574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd10ef-8914-40c8-b574-bef2f3c3d5b5 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:17.796149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd10ef-8914-40c8-b574-bef2f3c3d5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:17.796149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd10ef-8914-40c8-b574-bef2f3c3d5b5 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:17.797525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 699/1377] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:17.802714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebe8f-2fc8-4bbc-819a-6988b82801c5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:17.803166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebe8f-2fc8-4bbc-819a-6988b82801c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:17.803308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ebe8f-2fc8-4bbc-819a-6988b82801c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:17.821029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-306ebe8f-2fc8-4bbc-819a-6988b82801c5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:17.822270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-306ebe8f-2fc8-4bbc-819a-6988b82801c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:17.830739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 679/1378] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:17.875278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd10ef-8914-40c8-b574-bef2f3c3d5b5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:17.890048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd10ef-8914-40c8-b574-bef2f3c3d5b5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:17.896261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609c3a3-af0e-4a6a-926e-0422cbf2664d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:17.896993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609c3a3-af0e-4a6a-926e-0422cbf2664d service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:17.897282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609c3a3-af0e-4a6a-926e-0422cbf2664d service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:17.897556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609c3a3-af0e-4a6a-926e-0422cbf2664d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:17.906150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609c3a3-af0e-4a6a-926e-0422cbf2664d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:17.907737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609c3a3-af0e-4a6a-926e-0422cbf2664d service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:17.911724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:31:17.918430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 700/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:20.128153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc872c55-7c1f-47fc-8181-410489f852ac None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:20.128670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc872c55-7c1f-47fc-8181-410489f852ac None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:20.128781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc872c55-7c1f-47fc-8181-410489f852ac None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:20.130128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 680/1380] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:20.135368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891705c9-4996-421f-a270-12d787784880 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:20.136195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891705c9-4996-421f-a270-12d787784880 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:20.136479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891705c9-4996-421f-a270-12d787784880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:20.155420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-891705c9-4996-421f-a270-12d787784880 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:20.156602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-891705c9-4996-421f-a270-12d787784880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:20.164710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 701/1381] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:20.209060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc872c55-7c1f-47fc-8181-410489f852ac None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:20.224444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc872c55-7c1f-47fc-8181-410489f852ac None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:20.230648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09a522-cbc2-44e8-a7f7-b7d3efaacc8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:20.231464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09a522-cbc2-44e8-a7f7-b7d3efaacc8d service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:20.231742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09a522-cbc2-44e8-a7f7-b7d3efaacc8d service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:20.232014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09a522-cbc2-44e8-a7f7-b7d3efaacc8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:20.240798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09a522-cbc2-44e8-a7f7-b7d3efaacc8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:20.242410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09a522-cbc2-44e8-a7f7-b7d3efaacc8d service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:20.252821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 681/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:22.431541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace305a-7034-461e-a54e-0fb11263b18d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:22.432025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace305a-7034-461e-a54e-0fb11263b18d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:22.432129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace305a-7034-461e-a54e-0fb11263b18d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:22.433571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 702/1383] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:22.439563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7919a-b771-4bd2-a5a1-e5668910efc1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:22.440110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7919a-b771-4bd2-a5a1-e5668910efc1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:22.440359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7919a-b771-4bd2-a5a1-e5668910efc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:22.458912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ede7919a-b771-4bd2-a5a1-e5668910efc1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:22.460205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ede7919a-b771-4bd2-a5a1-e5668910efc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:22.468890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 682/1384] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:31:22.510980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace305a-7034-461e-a54e-0fb11263b18d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:22.526318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace305a-7034-461e-a54e-0fb11263b18d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:22.532205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2e53d-dc9f-4a9c-a81b-b51d2b1f63c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:22.532931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2e53d-dc9f-4a9c-a81b-b51d2b1f63c5 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:22.533335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2e53d-dc9f-4a9c-a81b-b51d2b1f63c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:22.533677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2e53d-dc9f-4a9c-a81b-b51d2b1f63c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:22.543663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2e53d-dc9f-4a9c-a81b-b51d2b1f63c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:22.545227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2e53d-dc9f-4a9c-a81b-b51d2b1f63c5 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:22.555371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 703/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:24.687349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7997-d13e-497f-8831-ae47355c6515 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:24.688021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7997-d13e-497f-8831-ae47355c6515 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:24.688021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7997-d13e-497f-8831-ae47355c6515 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:24.689381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 683/1386] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:24.695050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f521ca-3f89-463e-a3f4-f357e79d4467 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:24.695492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f521ca-3f89-463e-a3f4-f357e79d4467 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:24.695590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f521ca-3f89-463e-a3f4-f357e79d4467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:24.717768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-44f521ca-3f89-463e-a3f4-f357e79d4467 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:24.719033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-44f521ca-3f89-463e-a3f4-f357e79d4467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:24.727146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 704/1387] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:24.773540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a7997-d13e-497f-8831-ae47355c6515 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:24.788359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a7997-d13e-497f-8831-ae47355c6515 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:24.793890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a1d0a-a98c-4137-baf1-758255f8ba33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:24.794544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a1d0a-a98c-4137-baf1-758255f8ba33 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:24.794829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a1d0a-a98c-4137-baf1-758255f8ba33 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:24.795106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a1d0a-a98c-4137-baf1-758255f8ba33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:24.803276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a1d0a-a98c-4137-baf1-758255f8ba33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:24.804860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a1d0a-a98c-4137-baf1-758255f8ba33 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:24.814811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 684/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:26.999469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca10df-a283-49ff-b9d2-f75295636599 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:26.999988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca10df-a283-49ff-b9d2-f75295636599 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:27.000157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca10df-a283-49ff-b9d2-f75295636599 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:27.001889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 705/1389] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:27.007525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f046d5-949a-40fe-b11c-b6a857a0c950 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:27.007969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f046d5-949a-40fe-b11c-b6a857a0c950 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:27.008165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f046d5-949a-40fe-b11c-b6a857a0c950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:27.026867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c1f046d5-949a-40fe-b11c-b6a857a0c950 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:27.028345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c1f046d5-949a-40fe-b11c-b6a857a0c950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:27.036278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 685/1390] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:27.081502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca10df-a283-49ff-b9d2-f75295636599 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:27.096966 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca10df-a283-49ff-b9d2-f75295636599 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:27.102888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ed785-8fb9-41d3-9a8c-4bee4b7ff045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:27.103657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ed785-8fb9-41d3-9a8c-4bee4b7ff045 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:27.103941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ed785-8fb9-41d3-9a8c-4bee4b7ff045 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:27.104251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ed785-8fb9-41d3-9a8c-4bee4b7ff045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:27.113609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ed785-8fb9-41d3-9a8c-4bee4b7ff045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:27.115342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ed785-8fb9-41d3-9a8c-4bee4b7ff045 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:27.126641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 706/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:29.691188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bd467-1083-4c49-aa24-33967115d5c6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:29.691749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bd467-1083-4c49-aa24-33967115d5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:29.691749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bd467-1083-4c49-aa24-33967115d5c6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:29.693057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 686/1392] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:29.699018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ade682-0ba3-4743-8762-8a19180e6c81 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:29.699471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ade682-0ba3-4743-8762-8a19180e6c81 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:29.699581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ade682-0ba3-4743-8762-8a19180e6c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:29.719409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-54ade682-0ba3-4743-8762-8a19180e6c81 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:29.720612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-54ade682-0ba3-4743-8762-8a19180e6c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:29.728788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 707/1393] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:29.773886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bd467-1083-4c49-aa24-33967115d5c6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:29.789607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bd467-1083-4c49-aa24-33967115d5c6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:29.795533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eced7-5ec1-4b8e-9917-f2d0c98a35cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:29.796221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eced7-5ec1-4b8e-9917-f2d0c98a35cf service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:29.796508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eced7-5ec1-4b8e-9917-f2d0c98a35cf service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:29.796792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eced7-5ec1-4b8e-9917-f2d0c98a35cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:29.805201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eced7-5ec1-4b8e-9917-f2d0c98a35cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:29.806816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eced7-5ec1-4b8e-9917-f2d0c98a35cf service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:29.816320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 687/1394] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:32.590000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386199f4-c64d-40c4-936b-b06878fc0daf None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:32.590786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386199f4-c64d-40c4-936b-b06878fc0daf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:32.590786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386199f4-c64d-40c4-936b-b06878fc0daf None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:32.592040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 708/1395] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:32.598466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2148-23be-4b32-9308-2fbfb2bc5dcf None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:32.598929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2148-23be-4b32-9308-2fbfb2bc5dcf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:32.599284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2148-23be-4b32-9308-2fbfb2bc5dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:32.627502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-917d2148-23be-4b32-9308-2fbfb2bc5dcf None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:32.629297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-917d2148-23be-4b32-9308-2fbfb2bc5dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:32.641502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 688/1396] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:32.692325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386199f4-c64d-40c4-936b-b06878fc0daf None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:32.707536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386199f4-c64d-40c4-936b-b06878fc0daf None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:32.713523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5e765-f0c5-4bdd-982e-9a5325749fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:32.715277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5e765-f0c5-4bdd-982e-9a5325749fef service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:32.715587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5e765-f0c5-4bdd-982e-9a5325749fef service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:32.715861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5e765-f0c5-4bdd-982e-9a5325749fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:32.725203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5e765-f0c5-4bdd-982e-9a5325749fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:32.726889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5e765-f0c5-4bdd-982e-9a5325749fef service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:32.737813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 709/1397] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:34.954896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532957-1404-412b-8a1d-ff1e8b9252bb None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:34.955448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532957-1404-412b-8a1d-ff1e8b9252bb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:34.955448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532957-1404-412b-8a1d-ff1e8b9252bb None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:34.956789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 689/1398] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:34.962035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f65aa3-ba03-4d17-a7ac-8150e2188db1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:34.962462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f65aa3-ba03-4d17-a7ac-8150e2188db1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:34.962600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f65aa3-ba03-4d17-a7ac-8150e2188db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:34.980578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-25f65aa3-ba03-4d17-a7ac-8150e2188db1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:34.981770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-25f65aa3-ba03-4d17-a7ac-8150e2188db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:34.989521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 710/1399] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:35.032790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35532957-1404-412b-8a1d-ff1e8b9252bb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:35.041302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=55 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:31:35.049088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35532957-1404-412b-8a1d-ff1e8b9252bb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:35.054585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb32607-f309-41b2-b926-b6fbc4a4e73e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:35.055237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb32607-f309-41b2-b926-b6fbc4a4e73e service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:35.055553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb32607-f309-41b2-b926-b6fbc4a4e73e service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:35.056463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb32607-f309-41b2-b926-b6fbc4a4e73e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:35.064876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb32607-f309-41b2-b926-b6fbc4a4e73e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:35.066499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb32607-f309-41b2-b926-b6fbc4a4e73e service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:35.075949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 690/1400] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:36.521503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116d915-36cf-49bd-bea7-ffdc69e9f86d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:36.522094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116d915-36cf-49bd-bea7-ffdc69e9f86d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:36.522094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116d915-36cf-49bd-bea7-ffdc69e9f86d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:36.523468 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 711/1401] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:36.529206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b30d9-6023-4348-b038-88969aaa9f5e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:36.529624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b30d9-6023-4348-b038-88969aaa9f5e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:36.529762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b30d9-6023-4348-b038-88969aaa9f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:36.548107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-003b30d9-6023-4348-b038-88969aaa9f5e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:36.549349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-003b30d9-6023-4348-b038-88969aaa9f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:36.557848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 691/1402] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:36.614218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06671713-e3b3-4964-a99d-77e0efe17576 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:36.614579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06671713-e3b3-4964-a99d-77e0efe17576 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:36.614863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06671713-e3b3-4964-a99d-77e0efe17576 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:36.616241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 712/1403] 10.222.0.25 () {54 vars in 962 bytes} [Fri Jun 17 11:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:36.621055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44ae7d-58e4-443f-a154-60ab44a56e41 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:36.621345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44ae7d-58e4-443f-a154-60ab44a56e41 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:36.621621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44ae7d-58e4-443f-a154-60ab44a56e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:36.670036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ce44ae7d-58e4-443f-a154-60ab44a56e41 None None] MFA Rules not processed for user `86f3b6ba74fb49a280735bb8c867d26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:36.671239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ce44ae7d-58e4-443f-a154-60ab44a56e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:36.729742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 692/1404] 10.222.0.25 () {58 vars in 1059 bytes} [Fri Jun 17 11:31:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:36.737278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06671713-e3b3-4964-a99d-77e0efe17576 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:36.747833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06671713-e3b3-4964-a99d-77e0efe17576 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:36.751354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b863e-67df-42ab-9dd3-3ef34546f3ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f3b6ba74fb49a280735bb8c867d26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:36.751992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b863e-67df-42ab-9dd3-3ef34546f3ce service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:36.752278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b863e-67df-42ab-9dd3-3ef34546f3ce service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:36.752556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b863e-67df-42ab-9dd3-3ef34546f3ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:36.761530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b863e-67df-42ab-9dd3-3ef34546f3ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:36.763000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b863e-67df-42ab-9dd3-3ef34546f3ce service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:36.772297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 713/1405] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:31:37.468098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cefed6-9c29-4ea9-abf4-04fb7747b94e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:37.468389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cefed6-9c29-4ea9-abf4-04fb7747b94e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:37.468671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cefed6-9c29-4ea9-abf4-04fb7747b94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:37.516229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2cefed6-9c29-4ea9-abf4-04fb7747b94e None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:37.517434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2cefed6-9c29-4ea9-abf4-04fb7747b94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:37.579684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 693/1406] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:31:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:37.619442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabea243-e602-4777-86ce-d09659dd0bdc None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:37.619894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabea243-e602-4777-86ce-d09659dd0bdc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:37.620001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabea243-e602-4777-86ce-d09659dd0bdc None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:37.621392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 714/1407] 10.222.0.25 () {54 vars in 942 bytes} [Fri Jun 17 11:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:31:37.627806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e650-75af-4273-99a8-22bf6d2d5881 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:37.628099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e650-75af-4273-99a8-22bf6d2d5881 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:37.628379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e650-75af-4273-99a8-22bf6d2d5881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:37.673652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:31:37.676764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-81b7e650-75af-4273-99a8-22bf6d2d5881 None None] MFA Rules not processed for user `fd3119ad342840ddb0258e124be55597`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:31:37.678116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-81b7e650-75af-4273-99a8-22bf6d2d5881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:31:37.750275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 694/1408] 10.222.0.25 () {58 vars in 1039 bytes} [Fri Jun 17 11:31:37 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:31:37.759799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabea243-e602-4777-86ce-d09659dd0bdc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:31:37.773112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabea243-e602-4777-86ce-d09659dd0bdc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:31:37.778302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41172e60-084d-4f6e-85fd-9a4dfbcdcb8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:31:37.778954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41172e60-084d-4f6e-85fd-9a4dfbcdcb8c service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:31:37.779243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41172e60-084d-4f6e-85fd-9a4dfbcdcb8c service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:31:37.779524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41172e60-084d-4f6e-85fd-9a4dfbcdcb8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:31:37.788352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41172e60-084d-4f6e-85fd-9a4dfbcdcb8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:31:37.790524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41172e60-084d-4f6e-85fd-9a4dfbcdcb8c service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:31:37.799704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 715/1409] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:31:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:31:47.747302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=30 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:31:47.795894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=43 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:00.543021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c2e1e-4775-4bf5-af7f-d300cfadefef None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:00.543644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c2e1e-4775-4bf5-af7f-d300cfadefef None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:00.543644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c2e1e-4775-4bf5-af7f-d300cfadefef None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:00.544969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 695/1410] 10.222.0.25 () {54 vars in 918 bytes} [Fri Jun 17 11:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:00.550038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864baa-a5ba-46c8-b2f2-a3c93cc9ae83 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:00.550513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864baa-a5ba-46c8-b2f2-a3c93cc9ae83 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:00.550619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38864baa-a5ba-46c8-b2f2-a3c93cc9ae83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:00.568123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-38864baa-a5ba-46c8-b2f2-a3c93cc9ae83 None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:00.569362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-38864baa-a5ba-46c8-b2f2-a3c93cc9ae83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:00.578406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 716/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Jun 17 11:32:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:00.592901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45eb57-3f9c-4aba-a06b-9aeefcc643a6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:00.593196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45eb57-3f9c-4aba-a06b-9aeefcc643a6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:00.593499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45eb57-3f9c-4aba-a06b-9aeefcc643a6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:00.594882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 696/1412] 10.222.0.25 () {54 vars in 918 bytes} [Fri Jun 17 11:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:00.599207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fc0a-af62-4907-8d62-3e239d2da8a2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:00.599499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fc0a-af62-4907-8d62-3e239d2da8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:00.599780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fc0a-af62-4907-8d62-3e239d2da8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:00.617527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-91e6fc0a-af62-4907-8d62-3e239d2da8a2 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:00.618750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-91e6fc0a-af62-4907-8d62-3e239d2da8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:00.628234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 717/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Fri Jun 17 11:32:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:03.230346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242dea-f5ad-4096-bf21-b90c6b6f0ecb None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:03.230908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242dea-f5ad-4096-bf21-b90c6b6f0ecb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:03.230908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242dea-f5ad-4096-bf21-b90c6b6f0ecb None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:03.232283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:03.237626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9503f1-9c2c-4609-95b9-b303e816a182 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:03.238062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9503f1-9c2c-4609-95b9-b303e816a182 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:03.238302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9503f1-9c2c-4609-95b9-b303e816a182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:03.255879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fe9503f1-9c2c-4609-95b9-b303e816a182 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:03.257076 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fe9503f1-9c2c-4609-95b9-b303e816a182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:03.265384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:03.308519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1242dea-f5ad-4096-bf21-b90c6b6f0ecb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:03.323612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1242dea-f5ad-4096-bf21-b90c6b6f0ecb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:03.328936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcc745-530b-463a-9ef7-6aad7199c1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:03.329578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcc745-530b-463a-9ef7-6aad7199c1f7 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:03.329886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcc745-530b-463a-9ef7-6aad7199c1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:03.330169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcc745-530b-463a-9ef7-6aad7199c1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:03.338969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcc745-530b-463a-9ef7-6aad7199c1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:03.340575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcc745-530b-463a-9ef7-6aad7199c1f7 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:03.350062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 698/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:04.293647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f06a-85e2-43c6-a656-8b0442379d1d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:04.294260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f06a-85e2-43c6-a656-8b0442379d1d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:04.294260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f06a-85e2-43c6-a656-8b0442379d1d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:04.295623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:04.300418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3fc52-0027-45d6-9b34-fc94d2abfa63 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:04.300866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3fc52-0027-45d6-9b34-fc94d2abfa63 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:04.300974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3fc52-0027-45d6-9b34-fc94d2abfa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:04.319544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-89b3fc52-0027-45d6-9b34-fc94d2abfa63 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:04.320743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-89b3fc52-0027-45d6-9b34-fc94d2abfa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:04.328575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:04.499155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0f06a-85e2-43c6-a656-8b0442379d1d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:04.513211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0f06a-85e2-43c6-a656-8b0442379d1d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:04.518635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97864b-7182-490c-b5aa-c0aca2ab3ee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:04.519265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97864b-7182-490c-b5aa-c0aca2ab3ee0 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:04.519546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97864b-7182-490c-b5aa-c0aca2ab3ee0 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:04.519820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97864b-7182-490c-b5aa-c0aca2ab3ee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:04.528710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97864b-7182-490c-b5aa-c0aca2ab3ee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:04.530325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97864b-7182-490c-b5aa-c0aca2ab3ee0 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:04.540070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 720/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:05.456858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88c55-0957-4ca5-bd39-caedcdb95293 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:05.457385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88c55-0957-4ca5-bd39-caedcdb95293 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:05.457385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e88c55-0957-4ca5-bd39-caedcdb95293 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:05.458730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:05.463650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f637905-c189-4f60-a3e0-bf75bb9b8f58 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:05.464052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f637905-c189-4f60-a3e0-bf75bb9b8f58 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:05.464209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f637905-c189-4f60-a3e0-bf75bb9b8f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:05.482904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1f637905-c189-4f60-a3e0-bf75bb9b8f58 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:05.484094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1f637905-c189-4f60-a3e0-bf75bb9b8f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:05.492706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:05.643916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e88c55-0957-4ca5-bd39-caedcdb95293 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:05.658325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e88c55-0957-4ca5-bd39-caedcdb95293 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:05.664104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78636a06-ae3d-46cc-9a2b-8846ab571cd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:05.664743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78636a06-ae3d-46cc-9a2b-8846ab571cd2 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:05.665028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78636a06-ae3d-46cc-9a2b-8846ab571cd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:05.665308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78636a06-ae3d-46cc-9a2b-8846ab571cd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:05.673388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78636a06-ae3d-46cc-9a2b-8846ab571cd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:05.675012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78636a06-ae3d-46cc-9a2b-8846ab571cd2 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:05.685026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 701/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:06.598235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab531b0-61df-4344-8926-2b69712ab437 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:06.598795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab531b0-61df-4344-8926-2b69712ab437 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:06.598795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab531b0-61df-4344-8926-2b69712ab437 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:06.600170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:06.605111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb61a-f4ce-430e-ba33-9c307f9543d0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:06.605518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb61a-f4ce-430e-ba33-9c307f9543d0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:06.605676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb61a-f4ce-430e-ba33-9c307f9543d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:06.624992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-651bb61a-f4ce-430e-ba33-9c307f9543d0 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:06.626241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-651bb61a-f4ce-430e-ba33-9c307f9543d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:06.635094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:06.793442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab531b0-61df-4344-8926-2b69712ab437 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:06.808180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab531b0-61df-4344-8926-2b69712ab437 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:06.813471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1312ae-0a4d-4359-8927-6ef5fb47930a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:06.814142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1312ae-0a4d-4359-8927-6ef5fb47930a service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:06.814430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1312ae-0a4d-4359-8927-6ef5fb47930a service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:06.814707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1312ae-0a4d-4359-8927-6ef5fb47930a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:06.823275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1312ae-0a4d-4359-8927-6ef5fb47930a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:06.824848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1312ae-0a4d-4359-8927-6ef5fb47930a service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:06.834389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 723/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:07.737126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670f8d1-0702-4538-8e52-5fe1b7da9324 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:07.737654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670f8d1-0702-4538-8e52-5fe1b7da9324 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:07.737654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670f8d1-0702-4538-8e52-5fe1b7da9324 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:07.739076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:07.743854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f9d2b-a8d5-49f1-94b7-7591950e7914 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:07.744292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f9d2b-a8d5-49f1-94b7-7591950e7914 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:07.744420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f9d2b-a8d5-49f1-94b7-7591950e7914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:07.762776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a29f9d2b-a8d5-49f1-94b7-7591950e7914 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:07.763993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a29f9d2b-a8d5-49f1-94b7-7591950e7914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:07.771776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:07.923686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670f8d1-0702-4538-8e52-5fe1b7da9324 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:07.937600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670f8d1-0702-4538-8e52-5fe1b7da9324 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:07.942906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b7d98-5e0c-47ba-9c44-e518bc31b887 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:07.943543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b7d98-5e0c-47ba-9c44-e518bc31b887 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:07.943852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b7d98-5e0c-47ba-9c44-e518bc31b887 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:07.944124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b7d98-5e0c-47ba-9c44-e518bc31b887 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:07.952594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b7d98-5e0c-47ba-9c44-e518bc31b887 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:07.954216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b7d98-5e0c-47ba-9c44-e518bc31b887 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:07.964034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 704/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:08.813828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe640b-4d84-4f9d-85a0-15f4af48157f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:08.814384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe640b-4d84-4f9d-85a0-15f4af48157f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:08.814544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe640b-4d84-4f9d-85a0-15f4af48157f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:08.816042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:08.821486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313173-e557-409e-8fa7-4788c25b2bea None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:08.821931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313173-e557-409e-8fa7-4788c25b2bea None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:08.822149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313173-e557-409e-8fa7-4788c25b2bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:08.840263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ba313173-e557-409e-8fa7-4788c25b2bea None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:08.841458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ba313173-e557-409e-8fa7-4788c25b2bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:08.849182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:09.002699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe640b-4d84-4f9d-85a0-15f4af48157f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:09.018020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe640b-4d84-4f9d-85a0-15f4af48157f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:09.023510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4d41-51f4-452c-ba2b-87f6fd2ea173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:09.024149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4d41-51f4-452c-ba2b-87f6fd2ea173 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:09.024435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4d41-51f4-452c-ba2b-87f6fd2ea173 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:09.024713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4d41-51f4-452c-ba2b-87f6fd2ea173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:09.033087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd4d41-51f4-452c-ba2b-87f6fd2ea173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:09.034702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd4d41-51f4-452c-ba2b-87f6fd2ea173 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:09.044095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 726/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:09.890132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3269da-914c-4983-ab5e-3863e72b7428 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:09.890609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3269da-914c-4983-ab5e-3863e72b7428 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:09.890695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3269da-914c-4983-ab5e-3863e72b7428 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:09.892072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:32:09.897194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be68fd-2e93-4547-9f15-32557f41ae32 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:09.897671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be68fd-2e93-4547-9f15-32557f41ae32 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:09.897776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be68fd-2e93-4547-9f15-32557f41ae32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:09.915865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-38be68fd-2e93-4547-9f15-32557f41ae32 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:09.917083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-38be68fd-2e93-4547-9f15-32557f41ae32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:09.924889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:10.075053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3269da-914c-4983-ab5e-3863e72b7428 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:10.089103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3269da-914c-4983-ab5e-3863e72b7428 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:10.094399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de1df9-bbd7-4395-ade7-07088c95f862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:10.095043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1df9-bbd7-4395-ade7-07088c95f862 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:10.095325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1df9-bbd7-4395-ade7-07088c95f862 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:10.095601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de1df9-bbd7-4395-ade7-07088c95f862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:10.104117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de1df9-bbd7-4395-ade7-07088c95f862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:10.105640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de1df9-bbd7-4395-ade7-07088c95f862 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:10.114823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 707/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:10.959482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13543ded-0b1b-4058-a79b-b0cfdf22e19a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:10.960016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13543ded-0b1b-4058-a79b-b0cfdf22e19a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:10.960016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13543ded-0b1b-4058-a79b-b0cfdf22e19a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:10.961386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:10.966321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac57620-1ca8-493e-b84e-861df6120fde None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:10.966778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac57620-1ca8-493e-b84e-861df6120fde None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:10.966880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac57620-1ca8-493e-b84e-861df6120fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:10.985325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9ac57620-1ca8-493e-b84e-861df6120fde None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:10.986533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9ac57620-1ca8-493e-b84e-861df6120fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:10.994673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:11.144797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13543ded-0b1b-4058-a79b-b0cfdf22e19a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:11.159971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13543ded-0b1b-4058-a79b-b0cfdf22e19a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:11.165967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99906ba-9e6d-4226-a50e-821cb1646b69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:11.166736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99906ba-9e6d-4226-a50e-821cb1646b69 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:11.167023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99906ba-9e6d-4226-a50e-821cb1646b69 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:11.167303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99906ba-9e6d-4226-a50e-821cb1646b69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:11.175338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99906ba-9e6d-4226-a50e-821cb1646b69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:11.176934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99906ba-9e6d-4226-a50e-821cb1646b69 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:11.186308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 729/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:12.045639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4a3a-a227-4df5-a269-cdcffe25e98d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:12.046249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4a3a-a227-4df5-a269-cdcffe25e98d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:12.046249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4a3a-a227-4df5-a269-cdcffe25e98d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:12.047514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:12.052256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9208fc5-0b8b-40eb-b0a1-a55ef4c8a59d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:12.052708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9208fc5-0b8b-40eb-b0a1-a55ef4c8a59d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:12.052840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9208fc5-0b8b-40eb-b0a1-a55ef4c8a59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:12.071443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c9208fc5-0b8b-40eb-b0a1-a55ef4c8a59d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:12.072625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c9208fc5-0b8b-40eb-b0a1-a55ef4c8a59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:12.080315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:12.228758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed4a3a-a227-4df5-a269-cdcffe25e98d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:12.243277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed4a3a-a227-4df5-a269-cdcffe25e98d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:12.248553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8950f4-53c7-4ff0-9645-0a876c92d2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:12.249183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8950f4-53c7-4ff0-9645-0a876c92d2a6 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:12.249462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8950f4-53c7-4ff0-9645-0a876c92d2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:12.249731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8950f4-53c7-4ff0-9645-0a876c92d2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:12.257654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8950f4-53c7-4ff0-9645-0a876c92d2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:12.259244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8950f4-53c7-4ff0-9645-0a876c92d2a6 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:12.269470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 710/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:14.304038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad040e8-43ba-4579-9ebc-20c40ba2fbcc None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:14.304683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad040e8-43ba-4579-9ebc-20c40ba2fbcc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:14.304683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad040e8-43ba-4579-9ebc-20c40ba2fbcc None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:14.306051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:14.311274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f33f-8391-4c71-b82b-4c008a4ffa50 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:14.311725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f33f-8391-4c71-b82b-4c008a4ffa50 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:14.311840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f33f-8391-4c71-b82b-4c008a4ffa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:14.328992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5b38f33f-8391-4c71-b82b-4c008a4ffa50 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:14.330198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5b38f33f-8391-4c71-b82b-4c008a4ffa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:14.338338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:14.376366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad040e8-43ba-4579-9ebc-20c40ba2fbcc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:14.387306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad040e8-43ba-4579-9ebc-20c40ba2fbcc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:14.391199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25b6d5-3d3b-466f-9e69-5290267afb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:14.391840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25b6d5-3d3b-466f-9e69-5290267afb8c service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:14.392123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25b6d5-3d3b-466f-9e69-5290267afb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:14.392404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25b6d5-3d3b-466f-9e69-5290267afb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:14.401989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25b6d5-3d3b-466f-9e69-5290267afb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:14.403678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25b6d5-3d3b-466f-9e69-5290267afb8c service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:14.413194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 732/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:16.689175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665f9fe-15ff-4b90-a0f6-eb3569bd3138 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:16.689724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665f9fe-15ff-4b90-a0f6-eb3569bd3138 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:16.689724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665f9fe-15ff-4b90-a0f6-eb3569bd3138 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:16.691145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:16.697110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a72c00-263a-421a-997b-de285727a285 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:16.697540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a72c00-263a-421a-997b-de285727a285 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:16.697693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a72c00-263a-421a-997b-de285727a285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:16.715698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-43a72c00-263a-421a-997b-de285727a285 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:16.716891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-43a72c00-263a-421a-997b-de285727a285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:16.724571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:32:16.762838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665f9fe-15ff-4b90-a0f6-eb3569bd3138 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:16.773346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665f9fe-15ff-4b90-a0f6-eb3569bd3138 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:16.776952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c390c81-c54b-40d2-a1e1-4464c34ea07e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:16.777871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c390c81-c54b-40d2-a1e1-4464c34ea07e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:16.778157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c390c81-c54b-40d2-a1e1-4464c34ea07e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:16.778427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c390c81-c54b-40d2-a1e1-4464c34ea07e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:16.787133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c390c81-c54b-40d2-a1e1-4464c34ea07e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:16.788542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c390c81-c54b-40d2-a1e1-4464c34ea07e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:16.798494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 713/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:19.249774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17516bfe-b319-44d8-8637-34d4ac351f0d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:19.250352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17516bfe-b319-44d8-8637-34d4ac351f0d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:19.250352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17516bfe-b319-44d8-8637-34d4ac351f0d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:19.251781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:19.256977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31831ed9-5679-4395-b564-38d7a6601676 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:19.257381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31831ed9-5679-4395-b564-38d7a6601676 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:19.257598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31831ed9-5679-4395-b564-38d7a6601676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:19.275529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-31831ed9-5679-4395-b564-38d7a6601676 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:19.276721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-31831ed9-5679-4395-b564-38d7a6601676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:19.284900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:19.327514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17516bfe-b319-44d8-8637-34d4ac351f0d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:19.338896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17516bfe-b319-44d8-8637-34d4ac351f0d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:19.342344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac914a-421a-4530-8c49-ac77ee0288e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:19.342978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac914a-421a-4530-8c49-ac77ee0288e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:19.343256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac914a-421a-4530-8c49-ac77ee0288e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:19.343526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac914a-421a-4530-8c49-ac77ee0288e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:19.352737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac914a-421a-4530-8c49-ac77ee0288e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:19.354187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac914a-421a-4530-8c49-ac77ee0288e3 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:19.363302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 735/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:21.609050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eced93-94d0-4c6c-92c8-bf56157af241 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:21.609641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eced93-94d0-4c6c-92c8-bf56157af241 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:21.609641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eced93-94d0-4c6c-92c8-bf56157af241 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:21.611029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:21.618185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b37f-85d0-43e9-bac3-b14635bbc666 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:21.618619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b37f-85d0-43e9-bac3-b14635bbc666 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:21.618724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b37f-85d0-43e9-bac3-b14635bbc666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:21.637126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4235b37f-85d0-43e9-bac3-b14635bbc666 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:21.638377 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4235b37f-85d0-43e9-bac3-b14635bbc666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:21.646256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:21.684834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eced93-94d0-4c6c-92c8-bf56157af241 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:21.696155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eced93-94d0-4c6c-92c8-bf56157af241 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:21.699694 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827badb-8109-45c4-b9d6-26f28a461d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:21.700338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827badb-8109-45c4-b9d6-26f28a461d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:21.700622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827badb-8109-45c4-b9d6-26f28a461d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:21.700899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827badb-8109-45c4-b9d6-26f28a461d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:21.709786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827badb-8109-45c4-b9d6-26f28a461d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:21.711275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827badb-8109-45c4-b9d6-26f28a461d15 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:21.720572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 716/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:23.912020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce685e-785d-4df4-84a9-198594acc153 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:23.912544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce685e-785d-4df4-84a9-198594acc153 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:23.912544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce685e-785d-4df4-84a9-198594acc153 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:23.913927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:23.919191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddfd72-9244-4969-8276-9bbbbae640d9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:23.919644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddfd72-9244-4969-8276-9bbbbae640d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:23.919743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddfd72-9244-4969-8276-9bbbbae640d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:23.937607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0eddfd72-9244-4969-8276-9bbbbae640d9 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:23.938841 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0eddfd72-9244-4969-8276-9bbbbae640d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:23.947263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:32:23.984031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce685e-785d-4df4-84a9-198594acc153 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:23.995397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce685e-785d-4df4-84a9-198594acc153 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:23.998907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e278fb2-416f-4b79-a010-81d6f984e5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:23.999549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278fb2-416f-4b79-a010-81d6f984e5cd service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:23.999832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278fb2-416f-4b79-a010-81d6f984e5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:24.000109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e278fb2-416f-4b79-a010-81d6f984e5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:24.009466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e278fb2-416f-4b79-a010-81d6f984e5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:24.010917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e278fb2-416f-4b79-a010-81d6f984e5cd service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:24.020363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 738/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:26.187315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d505b0-8bd1-41a8-af1e-cabefe98c0f9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:26.187879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d505b0-8bd1-41a8-af1e-cabefe98c0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:26.187879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d505b0-8bd1-41a8-af1e-cabefe98c0f9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:26.189224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:26.194607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd794d21-e932-424e-9afd-90a6de6cc07f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:26.195024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd794d21-e932-424e-9afd-90a6de6cc07f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:26.195184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd794d21-e932-424e-9afd-90a6de6cc07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:26.212804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cd794d21-e932-424e-9afd-90a6de6cc07f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:26.214019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cd794d21-e932-424e-9afd-90a6de6cc07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:26.221581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:26.258309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d505b0-8bd1-41a8-af1e-cabefe98c0f9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:26.268881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d505b0-8bd1-41a8-af1e-cabefe98c0f9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:26.272404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efe66f-1216-4c10-ba01-4209777225cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:26.273039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe66f-1216-4c10-ba01-4209777225cd service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:26.273873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe66f-1216-4c10-ba01-4209777225cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:26.274156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efe66f-1216-4c10-ba01-4209777225cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:26.283212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efe66f-1216-4c10-ba01-4209777225cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:26.284640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efe66f-1216-4c10-ba01-4209777225cd service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:26.294603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 719/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:28.480729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b417-accf-4502-a287-9d9a57031915 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:28.481308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b417-accf-4502-a287-9d9a57031915 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:28.481308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321b417-accf-4502-a287-9d9a57031915 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:28.482704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:28.488011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607d140-30eb-4224-b08d-1fcbe26675fa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:28.488421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607d140-30eb-4224-b08d-1fcbe26675fa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:28.488564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607d140-30eb-4224-b08d-1fcbe26675fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:28.495121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=70 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:28.507488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a607d140-30eb-4224-b08d-1fcbe26675fa None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:28.509315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a607d140-30eb-4224-b08d-1fcbe26675fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:28.518787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:28.555917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321b417-accf-4502-a287-9d9a57031915 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:28.561303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=73 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:28.567723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321b417-accf-4502-a287-9d9a57031915 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:28.571865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c76dd-a8b0-436e-a6f5-bd046af09b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:28.572505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c76dd-a8b0-436e-a6f5-bd046af09b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:28.572797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c76dd-a8b0-436e-a6f5-bd046af09b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:28.573075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c76dd-a8b0-436e-a6f5-bd046af09b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:28.581962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c76dd-a8b0-436e-a6f5-bd046af09b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:28.583398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c76dd-a8b0-436e-a6f5-bd046af09b2f service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:28.592686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 741/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:30.745059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f594792-8142-4c5a-8c1d-27e16ecd1711 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:30.745583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f594792-8142-4c5a-8c1d-27e16ecd1711 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:30.745726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f594792-8142-4c5a-8c1d-27e16ecd1711 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:30.747016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:30.752165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693ca9-b70e-41bf-9c97-ce2a9ad0a3ae None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:30.752592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693ca9-b70e-41bf-9c97-ce2a9ad0a3ae None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:30.752734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693ca9-b70e-41bf-9c97-ce2a9ad0a3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:30.770795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-08693ca9-b70e-41bf-9c97-ce2a9ad0a3ae None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:30.771994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08693ca9-b70e-41bf-9c97-ce2a9ad0a3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:30.780022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:30.816741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f594792-8142-4c5a-8c1d-27e16ecd1711 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:30.827840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f594792-8142-4c5a-8c1d-27e16ecd1711 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:30.831304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f299e-b5ad-4fd6-953e-676a3c95ae97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:30.831933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f299e-b5ad-4fd6-953e-676a3c95ae97 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:30.832213 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f299e-b5ad-4fd6-953e-676a3c95ae97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:30.832483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f299e-b5ad-4fd6-953e-676a3c95ae97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:30.841178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f299e-b5ad-4fd6-953e-676a3c95ae97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:30.842626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f299e-b5ad-4fd6-953e-676a3c95ae97 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:30.851781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 722/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:32.994887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b7ad4-683a-4fbf-af90-10d41cc9eadb None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:32.995439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b7ad4-683a-4fbf-af90-10d41cc9eadb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:32.995439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b7ad4-683a-4fbf-af90-10d41cc9eadb None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:32.996826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:33.002060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c442751-3524-4132-a9c5-3f9b0962c554 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:33.002482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c442751-3524-4132-a9c5-3f9b0962c554 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:33.002623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c442751-3524-4132-a9c5-3f9b0962c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:33.020615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8c442751-3524-4132-a9c5-3f9b0962c554 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:33.021820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c442751-3524-4132-a9c5-3f9b0962c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:33.032291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:33.069417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b7ad4-683a-4fbf-af90-10d41cc9eadb None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:33.080652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b7ad4-683a-4fbf-af90-10d41cc9eadb None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:33.084105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39fa53-10d0-4187-9551-814845b1d293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:33.084740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fa53-10d0-4187-9551-814845b1d293 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:33.085023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fa53-10d0-4187-9551-814845b1d293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:33.085299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fa53-10d0-4187-9551-814845b1d293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:33.094085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39fa53-10d0-4187-9551-814845b1d293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:33.095502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39fa53-10d0-4187-9551-814845b1d293 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:33.104809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 744/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:35.236522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e299642-3a03-423c-b82a-2967178ab3f0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:35.237080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e299642-3a03-423c-b82a-2967178ab3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:35.237080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e299642-3a03-423c-b82a-2967178ab3f0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:35.238501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:35.244368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7fe4-3f77-416b-a28e-387b18a3f41a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:35.244813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7fe4-3f77-416b-a28e-387b18a3f41a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:35.244924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd7fe4-3f77-416b-a28e-387b18a3f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:35.263074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e1cd7fe4-3f77-416b-a28e-387b18a3f41a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:35.264275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e1cd7fe4-3f77-416b-a28e-387b18a3f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:35.272035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:35.308762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e299642-3a03-423c-b82a-2967178ab3f0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:35.319281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e299642-3a03-423c-b82a-2967178ab3f0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:35.323368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1400f0-82dc-431c-87e9-77a85f83e084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:35.324001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1400f0-82dc-431c-87e9-77a85f83e084 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:35.324281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1400f0-82dc-431c-87e9-77a85f83e084 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:35.324555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1400f0-82dc-431c-87e9-77a85f83e084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:35.333280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1400f0-82dc-431c-87e9-77a85f83e084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:35.334738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1400f0-82dc-431c-87e9-77a85f83e084 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:35.343954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 725/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:45.257314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=17 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:45.339821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=17 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:32:55.390676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfc983-f352-4200-8df8-c75429152ed5 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.391169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfc983-f352-4200-8df8-c75429152ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.391280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfc983-f352-4200-8df8-c75429152ed5 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.392658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 746/1471] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:55.397722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48df9-9dee-45b4-a3e6-8ce13d0a92a3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.398195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48df9-9dee-45b4-a3e6-8ce13d0a92a3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.398335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b48df9-9dee-45b4-a3e6-8ce13d0a92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.417555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-40b48df9-9dee-45b4-a3e6-8ce13d0a92a3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:55.418723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-40b48df9-9dee-45b4-a3e6-8ce13d0a92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:55.426527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 726/1472] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:55.472439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35510-5007-46c6-897e-a703c8817d4d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.472750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35510-5007-46c6-897e-a703c8817d4d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.473392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35510-5007-46c6-897e-a703c8817d4d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.474776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 747/1473] 10.222.0.25 () {54 vars in 962 bytes} [Fri Jun 17 11:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:55.480260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04f8f3-cf47-4a0b-96bb-a908f34e5050 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.480585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04f8f3-cf47-4a0b-96bb-a908f34e5050 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.480868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04f8f3-cf47-4a0b-96bb-a908f34e5050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.498629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2e04f8f3-cf47-4a0b-96bb-a908f34e5050 None None] MFA Rules not processed for user `86f3b6ba74fb49a280735bb8c867d26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:55.499831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2e04f8f3-cf47-4a0b-96bb-a908f34e5050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:55.506291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 727/1474] 10.222.0.25 () {58 vars in 1059 bytes} [Fri Jun 17 11:32:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:55.513637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35510-5007-46c6-897e-a703c8817d4d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:55.524709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35510-5007-46c6-897e-a703c8817d4d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:55.528211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c5f67-a805-44e6-a466-0db593245664 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f3b6ba74fb49a280735bb8c867d26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:55.528843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5f67-a805-44e6-a466-0db593245664 service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:55.529371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5f67-a805-44e6-a466-0db593245664 service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:55.529655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5f67-a805-44e6-a466-0db593245664 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:55.538391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c5f67-a805-44e6-a466-0db593245664 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:55.540479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c5f67-a805-44e6-a466-0db593245664 service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:55.552108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 748/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:32:57.748380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f30142-aba8-4871-904a-c6183d9b6995 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:57.748942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f30142-aba8-4871-904a-c6183d9b6995 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:57.748942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f30142-aba8-4871-904a-c6183d9b6995 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:57.750355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 728/1476] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:32:57.755597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e54725-e742-4364-b5aa-ba3ca4e4ea41 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:57.756041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e54725-e742-4364-b5aa-ba3ca4e4ea41 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:57.756171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e54725-e742-4364-b5aa-ba3ca4e4ea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:57.774273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f8e54725-e742-4364-b5aa-ba3ca4e4ea41 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:32:57.775490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f8e54725-e742-4364-b5aa-ba3ca4e4ea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:32:57.783743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 749/1477] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:32:57.807530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f30142-aba8-4871-904a-c6183d9b6995 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:32:57.820214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f30142-aba8-4871-904a-c6183d9b6995 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:32:57.825955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a82c7-2b2f-487a-8aec-c6b162c7db4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:32:57.826619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a82c7-2b2f-487a-8aec-c6b162c7db4b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:57.826905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a82c7-2b2f-487a-8aec-c6b162c7db4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:57.827185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a82c7-2b2f-487a-8aec-c6b162c7db4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:57.827925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a82c7-2b2f-487a-8aec-c6b162c7db4b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:32:57.829695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-162a82c7-2b2f-487a-8aec-c6b162c7db4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:32:57.829695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:32:57.829695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:32:57.830890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a82c7-2b2f-487a-8aec-c6b162c7db4b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:32:57.850357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 729/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Fri Jun 17 11:32:57 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:32:59.994456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08452e-8efa-4668-91f8-b04af148ef85 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:32:59.994995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08452e-8efa-4668-91f8-b04af148ef85 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:32:59.994995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08452e-8efa-4668-91f8-b04af148ef85 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:32:59.996401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 750/1479] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:00.001763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c53db5-3ef5-4752-807f-d6f45eddb57d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:00.002205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c53db5-3ef5-4752-807f-d6f45eddb57d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:00.002370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c53db5-3ef5-4752-807f-d6f45eddb57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:00.019596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e5c53db5-3ef5-4752-807f-d6f45eddb57d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:00.020774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e5c53db5-3ef5-4752-807f-d6f45eddb57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:00.029044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 730/1480] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:00.070939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08452e-8efa-4668-91f8-b04af148ef85 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:00.085070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08452e-8efa-4668-91f8-b04af148ef85 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:00.091000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976af50-603e-444a-8f22-0fe280ce3e6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:00.091648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976af50-603e-444a-8f22-0fe280ce3e6f service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:00.091935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976af50-603e-444a-8f22-0fe280ce3e6f service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:00.092214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976af50-603e-444a-8f22-0fe280ce3e6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:00.100335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976af50-603e-444a-8f22-0fe280ce3e6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:00.102575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976af50-603e-444a-8f22-0fe280ce3e6f service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:00.111898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 751/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:33:02.407636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346ccfa-9e09-4b72-bdd9-57c3b854be9a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:02.408175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346ccfa-9e09-4b72-bdd9-57c3b854be9a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:02.408175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346ccfa-9e09-4b72-bdd9-57c3b854be9a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:02.409570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 731/1482] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:33:02.414881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6f87e-817b-4e4e-8fa3-8ac33c438988 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:02.415357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6f87e-817b-4e4e-8fa3-8ac33c438988 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:02.415463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6f87e-817b-4e4e-8fa3-8ac33c438988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:02.433409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d2c6f87e-817b-4e4e-8fa3-8ac33c438988 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:02.434635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d2c6f87e-817b-4e4e-8fa3-8ac33c438988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:02.442316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 752/1483] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:02.478356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346ccfa-9e09-4b72-bdd9-57c3b854be9a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:02.489008 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346ccfa-9e09-4b72-bdd9-57c3b854be9a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:02.492524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23e058-da85-4a8b-a9f0-3eb19df60c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:02.493738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23e058-da85-4a8b-a9f0-3eb19df60c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:02.494049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23e058-da85-4a8b-a9f0-3eb19df60c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:02.494327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23e058-da85-4a8b-a9f0-3eb19df60c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:02.503087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23e058-da85-4a8b-a9f0-3eb19df60c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:02.504506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23e058-da85-4a8b-a9f0-3eb19df60c5d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:02.513778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 732/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:04.658708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581f34-3288-488b-8cfc-78f34d5fc415 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:04.659181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581f34-3288-488b-8cfc-78f34d5fc415 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:04.659373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581f34-3288-488b-8cfc-78f34d5fc415 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:04.660688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 753/1485] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:04.666626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3ec7e-b530-447a-9db4-be0f3637a459 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:04.667009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3ec7e-b530-447a-9db4-be0f3637a459 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:04.667182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3ec7e-b530-447a-9db4-be0f3637a459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:04.685574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2e3ec7e-b530-447a-9db4-be0f3637a459 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:04.686810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2e3ec7e-b530-447a-9db4-be0f3637a459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:04.694555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 733/1486] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:04.731985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d581f34-3288-488b-8cfc-78f34d5fc415 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:04.742685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d581f34-3288-488b-8cfc-78f34d5fc415 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:04.746218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891c701-2622-4ace-a80a-d887e7c4ed5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:04.746854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891c701-2622-4ace-a80a-d887e7c4ed5c service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:04.747143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891c701-2622-4ace-a80a-d887e7c4ed5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:04.747422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891c701-2622-4ace-a80a-d887e7c4ed5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:04.756441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891c701-2622-4ace-a80a-d887e7c4ed5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:04.757891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891c701-2622-4ace-a80a-d887e7c4ed5c service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:04.767766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 754/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:07.042517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438d7c9-700e-491e-a178-587b334ee30a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:07.043078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438d7c9-700e-491e-a178-587b334ee30a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:07.043078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438d7c9-700e-491e-a178-587b334ee30a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:07.044456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 734/1488] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:07.049705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067f73a-508f-423a-8345-e7877cbd1e7c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:07.050166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067f73a-508f-423a-8345-e7877cbd1e7c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:07.050300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067f73a-508f-423a-8345-e7877cbd1e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:07.069117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4067f73a-508f-423a-8345-e7877cbd1e7c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:07.070325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4067f73a-508f-423a-8345-e7877cbd1e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:07.078057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 755/1489] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:07.116983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438d7c9-700e-491e-a178-587b334ee30a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:07.128329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438d7c9-700e-491e-a178-587b334ee30a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:07.131752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc042ba-1da9-46e0-8a51-9605335f7c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:07.132387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc042ba-1da9-46e0-8a51-9605335f7c1a service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:07.132666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc042ba-1da9-46e0-8a51-9605335f7c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:07.132939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc042ba-1da9-46e0-8a51-9605335f7c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:07.141948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc042ba-1da9-46e0-8a51-9605335f7c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:07.143421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc042ba-1da9-46e0-8a51-9605335f7c1a service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:07.152639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 735/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:09.486885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ec978-4eae-4b39-86d1-77891114f821 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:09.487421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ec978-4eae-4b39-86d1-77891114f821 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:09.487421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ec978-4eae-4b39-86d1-77891114f821 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:09.488753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 756/1491] 10.222.0.25 () {54 vars in 926 bytes} [Fri Jun 17 11:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:33:09.494673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cd29a-523f-4c0a-8391-ad2b69ee9151 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:09.495097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cd29a-523f-4c0a-8391-ad2b69ee9151 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:09.495208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cd29a-523f-4c0a-8391-ad2b69ee9151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:09.512571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be5cd29a-523f-4c0a-8391-ad2b69ee9151 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:33:09.513768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be5cd29a-523f-4c0a-8391-ad2b69ee9151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:33:09.521584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 736/1492] 10.222.0.25 () {58 vars in 1023 bytes} [Fri Jun 17 11:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:33:09.560689 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ec978-4eae-4b39-86d1-77891114f821 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:33:09.571177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ec978-4eae-4b39-86d1-77891114f821 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:33:09.574611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878f8c3-d826-4d7b-aef1-8dcc75f20d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:33:09.575242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f8c3-d826-4d7b-aef1-8dcc75f20d6a service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:33:09.575520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f8c3-d826-4d7b-aef1-8dcc75f20d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:33:09.575789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f8c3-d826-4d7b-aef1-8dcc75f20d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:33:09.584312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878f8c3-d826-4d7b-aef1-8dcc75f20d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:33:09.585728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878f8c3-d826-4d7b-aef1-8dcc75f20d6a service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:33:09.595416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 757/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:33:19.507557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=33 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:33:19.591338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=33 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:34:49.933239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c28bf-827c-4b4b-b517-fdcca23314a7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:49.933812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c28bf-827c-4b4b-b517-fdcca23314a7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:49.933812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c28bf-827c-4b4b-b517-fdcca23314a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:49.951849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ec0c28bf-827c-4b4b-b517-fdcca23314a7 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:49.953073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ec0c28bf-827c-4b4b-b517-fdcca23314a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:49.960748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 737/1494] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:49.970991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8355aaa-7bff-4d5c-9d06-5e0ab2f24ecf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:49.971492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8355aaa-7bff-4d5c-9d06-5e0ab2f24ecf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:49.971608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8355aaa-7bff-4d5c-9d06-5e0ab2f24ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:49.991663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c8355aaa-7bff-4d5c-9d06-5e0ab2f24ecf None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:49.991836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c8355aaa-7bff-4d5c-9d06-5e0ab2f24ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:49.999523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 758/1495] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:50.004077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c28bf-827c-4b4b-b517-fdcca23314a7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.014171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c28bf-827c-4b4b-b517-fdcca23314a7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.018611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.019269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.019549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.019825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.023976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:34:50.023976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.023976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:50.023976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:34:50.023976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:34:50.023976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.035101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 738/1496] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:34:50.042986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8355aaa-7bff-4d5c-9d06-5e0ab2f24ecf None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.052152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8355aaa-7bff-4d5c-9d06-5e0ab2f24ecf None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.056990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.057647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.057980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.058258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.058963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.060730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:50.060730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 17 11:34:50.060730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: Jun 17 11:34:50.062022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.100519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 759/1497] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:34:50 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:34:50.102414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.112202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f512b5-11a1-4ca6-97b1-97702c99bb56 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.166557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.167222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.167506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.167783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.168483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.169492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:50.169492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:34:50.169492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:34:50.170728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.189946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 739/1498] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:34:50 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:34:50.194044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.204558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbaa76-658f-40a5-9dca-a646572d3521 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.208920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.209573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.209865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.210158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.210879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.211873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:50.211873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 17 11:34:50.211873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: Jun 17 11:34:50.212759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.225204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 760/1499] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:34:50.229203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.239167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852c65d-4e13-4207-b041-5481a71b7e14 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.244157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb3ae9-46b6-4eb4-b979-691aeb43ed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.244844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3ae9-46b6-4eb4-b979-691aeb43ed0c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.245137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3ae9-46b6-4eb4-b979-691aeb43ed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.245414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3ae9-46b6-4eb4-b979-691aeb43ed0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.246176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb3ae9-46b6-4eb4-b979-691aeb43ed0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.247201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-93eb3ae9-46b6-4eb4-b979-691aeb43ed0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:50.247201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:34:50.247201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:34:50.248087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb3ae9-46b6-4eb4-b979-691aeb43ed0c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.261865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 740/1500] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:34:50.265806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.275555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba269-9193-401f-ac7d-3efbce5505be admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.279970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b34718-e469-416f-832c-b49aa12a0ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.280837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b34718-e469-416f-832c-b49aa12a0ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.281115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b34718-e469-416f-832c-b49aa12a0ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.281387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b34718-e469-416f-832c-b49aa12a0ac6 admin admin] PATH_INFO: `/v3/projects/739ecaddc45f4a45bc1304a3c0b47152/users/9c38585be12a43a7bfc528ad22db1a2d/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.295941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b34718-e469-416f-832c-b49aa12a0ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=739ecaddc45f4a45bc1304a3c0b47152, user_id=9c38585be12a43a7bfc528ad22db1a2d, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.296948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-c4b34718-e469-416f-832c-b49aa12a0ac6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:50.296948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 17 11:34:50.296948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: Jun 17 11:34:50.298906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b34718-e469-416f-832c-b49aa12a0ac6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.311945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 761/1501] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:34:50 2022] PUT /identity/v3/projects/739ecaddc45f4a45bc1304a3c0b47152/users/9c38585be12a43a7bfc528ad22db1a2d/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:34:50.323560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf59ed8-286d-4247-a00d-b313e10fda47 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.323851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf59ed8-286d-4247-a00d-b313e10fda47 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.324139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf59ed8-286d-4247-a00d-b313e10fda47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.349473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1cf59ed8-286d-4247-a00d-b313e10fda47 None None] MFA Rules not processed for user `9c38585be12a43a7bfc528ad22db1a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:50.350709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1cf59ed8-286d-4247-a00d-b313e10fda47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:50.417441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 741/1502] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:34:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:50.430899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599ac03-b46b-4861-9a04-5abe550fa0cd None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.431194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599ac03-b46b-4861-9a04-5abe550fa0cd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.431475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599ac03-b46b-4861-9a04-5abe550fa0cd None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.432868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 762/1503] 10.222.0.25 () {54 vars in 963 bytes} [Fri Jun 17 11:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:50.437670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d38e-2eee-4727-9074-3677f2ef3497 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.438012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d38e-2eee-4727-9074-3677f2ef3497 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.438296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d38e-2eee-4727-9074-3677f2ef3497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.456693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8d10d38e-2eee-4727-9074-3677f2ef3497 None None] MFA Rules not processed for user `17154b0972cb4534a5c379babcb9d2f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:50.457827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8d10d38e-2eee-4727-9074-3677f2ef3497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:50.500877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:34:50.509436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 742/1504] 10.222.0.25 () {58 vars in 1060 bytes} [Fri Jun 17 11:34:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:50.516911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599ac03-b46b-4861-9a04-5abe550fa0cd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.527303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599ac03-b46b-4861-9a04-5abe550fa0cd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.530763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.531391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.531670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.531944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.540770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.542258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.550833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 763/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:34:50.592083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a87c1-83e0-433a-88fd-9f21dd91d1dd None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.592379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a87c1-83e0-433a-88fd-9f21dd91d1dd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.592661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a87c1-83e0-433a-88fd-9f21dd91d1dd None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.593990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 743/1506] 10.222.0.25 () {52 vars in 829 bytes} [Fri Jun 17 11:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:34:50.597538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.660077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dfa9c-f699-48e6-bdc3-667e00de5682 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.665232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.666183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.666463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.666733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] PATH_INFO: `/v3/users/9c38585be12a43a7bfc528ad22db1a2d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.667922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c38585be12a43a7bfc528ad22db1a2d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.668918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:50.668918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 17 11:34:50.668918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: Jun 17 11:34:50.670174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.732804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c38585be12a43a7bfc528ad22db1a2d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:50.741999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c38585be12a43a7bfc528ad22db1a2d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:50.754581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c38585be12a43a7bfc528ad22db1a2d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:50.755258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c38585be12a43a7bfc528ad22db1a2d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:50.771064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:34:50.773750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c38585be12a43a7bfc528ad22db1a2d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:50.781561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 764/1507] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:34:50 2022] DELETE /identity/v3/users/9c38585be12a43a7bfc528ad22db1a2d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:34:50.786219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.786513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.786797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.804614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:34:50.806477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:50.863970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 744/1508] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:34:50.873607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:50.875128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:50.932096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ab7b1-a9a1-4301-900d-104f2ce31c08 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:50.935776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dc74f-9a01-4232-b092-1f1cb6503bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:50.936460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dc74f-9a01-4232-b092-1f1cb6503bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:50.936745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dc74f-9a01-4232-b092-1f1cb6503bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:50.937022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dc74f-9a01-4232-b092-1f1cb6503bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:50.946023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dc74f-9a01-4232-b092-1f1cb6503bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:50.947459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dc74f-9a01-4232-b092-1f1cb6503bd8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:50.956819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 765/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:34:51.282884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:34:51.284899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:34:51.296572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0af746-6c40-4700-8153-4407818bc273 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:34:51.301139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:34:51.302914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:34:51.303149 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:34:51.303424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] PATH_INFO: `/v3/projects/739ecaddc45f4a45bc1304a3c0b47152` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:34:51.304623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=739ecaddc45f4a45bc1304a3c0b47152)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:34:51.305638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:34:51.305638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:34:51.305638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:34:51.308054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:34:51.368854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '739ecaddc45f4a45bc1304a3c0b47152', 'request_id': 'req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:51.377982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '739ecaddc45f4a45bc1304a3c0b47152', 'request_id': 'req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:51.384927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '739ecaddc45f4a45bc1304a3c0b47152', 'request_id': 'req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:51.403640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] The token cache is being invalidate because project 739ecaddc45f4a45bc1304a3c0b47152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:34:51.403929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:51.404684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8536d8a-020c-4f4e-b6d9-251cfd5bd9e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:34:51.406288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 745/1510] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:34:51 2022] DELETE /identity/v3/projects/739ecaddc45f4a45bc1304a3c0b47152 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:35:00.952798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:35:01.400519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:36:33.149808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c5b09-faae-4de9-8bca-73c39db99a37 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:33.150428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c5b09-faae-4de9-8bca-73c39db99a37 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:33.150428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c5b09-faae-4de9-8bca-73c39db99a37 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:33.151816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:36:33.156552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a4efd-9ea3-45c3-a6ae-1abbf68af0b2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:33.156998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a4efd-9ea3-45c3-a6ae-1abbf68af0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:33.157096 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a4efd-9ea3-45c3-a6ae-1abbf68af0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:33.175679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b13a4efd-9ea3-45c3-a6ae-1abbf68af0b2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:33.176893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b13a4efd-9ea3-45c3-a6ae-1abbf68af0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:33.234503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:33.405496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c5b09-faae-4de9-8bca-73c39db99a37 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:33.407381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d69c5b09-faae-4de9-8bca-73c39db99a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:33.484672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c5b09-faae-4de9-8bca-73c39db99a37 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:33.490085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558240f9-4f73-427e-828d-175142a3c3c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:33.490737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558240f9-4f73-427e-828d-175142a3c3c5 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:33.491021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558240f9-4f73-427e-828d-175142a3c3c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:33.491296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558240f9-4f73-427e-828d-175142a3c3c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:33.499256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558240f9-4f73-427e-828d-175142a3c3c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:33.500827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558240f9-4f73-427e-828d-175142a3c3c5 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:33.510692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 767/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:36:34.398729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb789b-deea-4832-a56b-706114af29fc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:34.399436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb789b-deea-4832-a56b-706114af29fc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:34.399436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb789b-deea-4832-a56b-706114af29fc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:34.400716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:36:34.405698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec714-b0fc-4420-97e1-ec4704871d32 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:34.406154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec714-b0fc-4420-97e1-ec4704871d32 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:34.406298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ec714-b0fc-4420-97e1-ec4704871d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:34.424534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3c1ec714-b0fc-4420-97e1-ec4704871d32 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:34.425735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3c1ec714-b0fc-4420-97e1-ec4704871d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:34.434236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:34.587363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb789b-deea-4832-a56b-706114af29fc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:34.601817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb789b-deea-4832-a56b-706114af29fc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:34.607730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa8ee8-8c1e-40bd-b6d8-643bdcaa514c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:34.608378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8ee8-8c1e-40bd-b6d8-643bdcaa514c service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:34.608666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8ee8-8c1e-40bd-b6d8-643bdcaa514c service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:34.608946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8ee8-8c1e-40bd-b6d8-643bdcaa514c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:34.616994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa8ee8-8c1e-40bd-b6d8-643bdcaa514c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:34.618643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa8ee8-8c1e-40bd-b6d8-643bdcaa514c service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:34.628195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 748/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:36:34.918123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cabe03-5cc4-4385-95ec-33a0af5feccb None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:34.918329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cabe03-5cc4-4385-95ec-33a0af5feccb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:34.918609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cabe03-5cc4-4385-95ec-33a0af5feccb None None] PATH_INFO: `/v3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:34.919930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 769/1517] 10.222.0.71 () {54 vars in 925 bytes} [Fri Jun 17 11:36:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:36:34.926618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7543b5-50a4-4b36-9b4a-774ba094a2ed None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:34.926908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7543b5-50a4-4b36-9b4a-774ba094a2ed None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:34.927185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7543b5-50a4-4b36-9b4a-774ba094a2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:34.944748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4d7543b5-50a4-4b36-9b4a-774ba094a2ed None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:34.946475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4d7543b5-50a4-4b36-9b4a-774ba094a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:35.011992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 749/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Jun 17 11:36:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:36:44.290166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4ef6-991c-4fbb-b910-6f197f1d65e3 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:44.290692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4ef6-991c-4fbb-b910-6f197f1d65e3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:44.290692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce4ef6-991c-4fbb-b910-6f197f1d65e3 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:44.292025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 770/1519] 10.222.0.25 () {54 vars in 960 bytes} [Fri Jun 17 11:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:36:44.296933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877332d4-c04c-47ab-90f8-5eb350895c15 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:44.297327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877332d4-c04c-47ab-90f8-5eb350895c15 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:44.297495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877332d4-c04c-47ab-90f8-5eb350895c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:44.315161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-877332d4-c04c-47ab-90f8-5eb350895c15 None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:44.316357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-877332d4-c04c-47ab-90f8-5eb350895c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:44.326520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 750/1520] 10.222.0.25 () {58 vars in 1057 bytes} [Fri Jun 17 11:36:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:44.334463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce4ef6-991c-4fbb-b910-6f197f1d65e3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:44.348569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce4ef6-991c-4fbb-b910-6f197f1d65e3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:44.354098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdfab8-ee16-40e8-842e-c1338fd33e3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:44.354739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdfab8-ee16-40e8-842e-c1338fd33e3e service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:44.355029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdfab8-ee16-40e8-842e-c1338fd33e3e service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:44.355309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdfab8-ee16-40e8-842e-c1338fd33e3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:44.363546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdfab8-ee16-40e8-842e-c1338fd33e3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:44.365136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdfab8-ee16-40e8-842e-c1338fd33e3e service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:44.375142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 771/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:36:45.939415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de67f-fd8f-4760-9484-fc898d195378 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:45.939957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de67f-fd8f-4760-9484-fc898d195378 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:45.939957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3de67f-fd8f-4760-9484-fc898d195378 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:45.941268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:36:45.946779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640bc97-b838-4ef4-a86c-3f7679ad3510 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:45.947190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640bc97-b838-4ef4-a86c-3f7679ad3510 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:45.947335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640bc97-b838-4ef4-a86c-3f7679ad3510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:45.965609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3640bc97-b838-4ef4-a86c-3f7679ad3510 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:45.966858 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3640bc97-b838-4ef4-a86c-3f7679ad3510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:45.974677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:46.127030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3de67f-fd8f-4760-9484-fc898d195378 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:46.141566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3de67f-fd8f-4760-9484-fc898d195378 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:46.146898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d3af-99bd-4647-9fd0-24c39f5f4187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:46.147540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d3af-99bd-4647-9fd0-24c39f5f4187 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:46.147816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d3af-99bd-4647-9fd0-24c39f5f4187 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:46.148088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d3af-99bd-4647-9fd0-24c39f5f4187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:46.156636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232d3af-99bd-4647-9fd0-24c39f5f4187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:46.158262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232d3af-99bd-4647-9fd0-24c39f5f4187 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:46.167797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 752/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:36:47.048812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2f75-4e02-4742-8e55-ade26ab9037b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:47.049374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2f75-4e02-4742-8e55-ade26ab9037b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:47.049374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a2f75-4e02-4742-8e55-ade26ab9037b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:47.050883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 17 11:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:36:47.056210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46e501-c178-4b72-8d52-62f5053c5f9d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:47.056653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46e501-c178-4b72-8d52-62f5053c5f9d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:47.056759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46e501-c178-4b72-8d52-62f5053c5f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:47.075004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7d46e501-c178-4b72-8d52-62f5053c5f9d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:47.076192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d46e501-c178-4b72-8d52-62f5053c5f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:47.083830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 17 11:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:47.233738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a2f75-4e02-4742-8e55-ade26ab9037b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:47.248656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a2f75-4e02-4742-8e55-ade26ab9037b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:47.254072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbdb8d-49fd-497a-955a-48805aef3b07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:47.254712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbdb8d-49fd-497a-955a-48805aef3b07 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:47.255002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbdb8d-49fd-497a-955a-48805aef3b07 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:47.255284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbdb8d-49fd-497a-955a-48805aef3b07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:47.263982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbdb8d-49fd-497a-955a-48805aef3b07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:47.265561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbdb8d-49fd-497a-955a-48805aef3b07 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:47.275100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 774/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:36:51.397251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4174-a984-40d5-9c26-925045529f1e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:51.397821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4174-a984-40d5-9c26-925045529f1e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:51.397821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4174-a984-40d5-9c26-925045529f1e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:51.399899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:36:51.404995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5b1f4-4957-4fe7-8725-591672f6bdc1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:51.405443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5b1f4-4957-4fe7-8725-591672f6bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:51.405574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5b1f4-4957-4fe7-8725-591672f6bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:51.423390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b7f5b1f4-4957-4fe7-8725-591672f6bdc1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:51.424672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b7f5b1f4-4957-4fe7-8725-591672f6bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:51.432698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:51.469197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4174-a984-40d5-9c26-925045529f1e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:51.470927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-184d4174-a984-40d5-9c26-925045529f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:51.527155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4174-a984-40d5-9c26-925045529f1e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:51.530619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe79bd-7da8-4fcb-a633-66735bbcc527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:51.531258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe79bd-7da8-4fcb-a633-66735bbcc527 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:51.531538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe79bd-7da8-4fcb-a633-66735bbcc527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:51.531812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe79bd-7da8-4fcb-a633-66735bbcc527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:51.540856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe79bd-7da8-4fcb-a633-66735bbcc527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:51.542243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe79bd-7da8-4fcb-a633-66735bbcc527 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:51.547071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:36:51.552022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 755/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:36:53.645423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e24aa3-095b-4010-bfa0-ff4e3c782d5b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:53.645863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e24aa3-095b-4010-bfa0-ff4e3c782d5b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:53.646024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e24aa3-095b-4010-bfa0-ff4e3c782d5b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:53.647422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:36:53.653189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa91ae-3dff-4b25-acc5-f8db6d2e438c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:53.653615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa91ae-3dff-4b25-acc5-f8db6d2e438c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:53.653753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa91ae-3dff-4b25-acc5-f8db6d2e438c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:53.671899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f0fa91ae-3dff-4b25-acc5-f8db6d2e438c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:53.673104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f0fa91ae-3dff-4b25-acc5-f8db6d2e438c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:53.681260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:53.716937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e24aa3-095b-4010-bfa0-ff4e3c782d5b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:53.727498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e24aa3-095b-4010-bfa0-ff4e3c782d5b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:53.730984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4987cbc-545c-4df4-a408-b70f19b64bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:53.731620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4987cbc-545c-4df4-a408-b70f19b64bad service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:53.731904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4987cbc-545c-4df4-a408-b70f19b64bad service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:53.732177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4987cbc-545c-4df4-a408-b70f19b64bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:53.740875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4987cbc-545c-4df4-a408-b70f19b64bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:53.742345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4987cbc-545c-4df4-a408-b70f19b64bad service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:53.752060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 777/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:36:57.558498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a180a0-90df-4c15-8594-13e6b26fa7b3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:57.559023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a180a0-90df-4c15-8594-13e6b26fa7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:57.559023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a180a0-90df-4c15-8594-13e6b26fa7b3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:57.560394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:36:57.565818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1ed3d-61f7-4ef7-bb1b-fa665c378051 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:57.566259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1ed3d-61f7-4ef7-bb1b-fa665c378051 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:57.566410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1ed3d-61f7-4ef7-bb1b-fa665c378051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:57.584331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-37a1ed3d-61f7-4ef7-bb1b-fa665c378051 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:36:57.585532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-37a1ed3d-61f7-4ef7-bb1b-fa665c378051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:57.593862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:36:57.861333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a180a0-90df-4c15-8594-13e6b26fa7b3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:36:57.863051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b1a180a0-90df-4c15-8594-13e6b26fa7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:36:57.919986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a180a0-90df-4c15-8594-13e6b26fa7b3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:36:57.923522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029bc9c-4a3c-4d11-9916-4e7425d035a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:36:57.924168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029bc9c-4a3c-4d11-9916-4e7425d035a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:36:57.924450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029bc9c-4a3c-4d11-9916-4e7425d035a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:36:57.924725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029bc9c-4a3c-4d11-9916-4e7425d035a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:36:57.933492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029bc9c-4a3c-4d11-9916-4e7425d035a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:36:57.934984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029bc9c-4a3c-4d11-9916-4e7425d035a0 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:36:57.944220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 758/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:37:00.084861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef80add-ed1b-48ac-a1e5-3b45ececa6f4 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:00.085440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef80add-ed1b-48ac-a1e5-3b45ececa6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:00.085440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef80add-ed1b-48ac-a1e5-3b45ececa6f4 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:00.086944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 17 11:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:37:00.092088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeacb0d-c3d4-4787-b46a-d2dbb51cf1a5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:00.092488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeacb0d-c3d4-4787-b46a-d2dbb51cf1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:00.092645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeacb0d-c3d4-4787-b46a-d2dbb51cf1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:00.110732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5eeacb0d-c3d4-4787-b46a-d2dbb51cf1a5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:37:00.111945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5eeacb0d-c3d4-4787-b46a-d2dbb51cf1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:00.120310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 17 11:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:37:00.383744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef80add-ed1b-48ac-a1e5-3b45ececa6f4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:37:00.385426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6ef80add-ed1b-48ac-a1e5-3b45ececa6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:37:00.398124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef80add-ed1b-48ac-a1e5-3b45ececa6f4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:37:00.401587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb93791-3251-40c8-974a-6ece46e87147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:37:00.402260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb93791-3251-40c8-974a-6ece46e87147 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:37:00.402547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb93791-3251-40c8-974a-6ece46e87147 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:37:00.402824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb93791-3251-40c8-974a-6ece46e87147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:37:00.412034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb93791-3251-40c8-974a-6ece46e87147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:37:00.413760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb93791-3251-40c8-974a-6ece46e87147 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:37:00.428564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 780/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:37:10.105496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=32 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:37:10.420628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=70 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:30.806391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65161ab0-5c5a-482c-8ff0-e6d24b0fe7e1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:30.807069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65161ab0-5c5a-482c-8ff0-e6d24b0fe7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:30.807069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65161ab0-5c5a-482c-8ff0-e6d24b0fe7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:30.844081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-65161ab0-5c5a-482c-8ff0-e6d24b0fe7e1 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:30.845303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-65161ab0-5c5a-482c-8ff0-e6d24b0fe7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:30.864992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 760/1540] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:30.877318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c7aab-4bc6-477d-b68c-c830c259d288 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:30.877968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c7aab-4bc6-477d-b68c-c830c259d288 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:30.877968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c7aab-4bc6-477d-b68c-c830c259d288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:30.915477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ed9c7aab-4bc6-477d-b68c-c830c259d288 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:30.916834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ed9c7aab-4bc6-477d-b68c-c830c259d288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:30.927375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 781/1541] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:38:30.931656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65161ab0-5c5a-482c-8ff0-e6d24b0fe7e1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:30.949991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65161ab0-5c5a-482c-8ff0-e6d24b0fe7e1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:30.955824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:30.956523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:30.956800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:30.957073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:30.957812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:30.958140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:30.959784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:30.980561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 761/1542] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:30.989271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c7aab-4bc6-477d-b68c-c830c259d288 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:30.999930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c7aab-4bc6-477d-b68c-c830c259d288 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.009924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea39e4-367b-4a11-a14c-6076566ad3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.011473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea39e4-367b-4a11-a14c-6076566ad3ca admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.011976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea39e4-367b-4a11-a14c-6076566ad3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.012466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea39e4-367b-4a11-a14c-6076566ad3ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.013388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea39e4-367b-4a11-a14c-6076566ad3ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.016113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea39e4-367b-4a11-a14c-6076566ad3ca admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.070605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 782/1543] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:30 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:31.075306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.090229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4743a9-1f29-49e8-b5a7-cd9672230d66 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.091828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7496f-82f8-46c8-ba7f-5d21a0e4b1f3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.092215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7496f-82f8-46c8-ba7f-5d21a0e4b1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.092752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7496f-82f8-46c8-ba7f-5d21a0e4b1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.134919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-feb7496f-82f8-46c8-ba7f-5d21a0e4b1f3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.137379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-feb7496f-82f8-46c8-ba7f-5d21a0e4b1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.178550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.179619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.179959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.180791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.181478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.184066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.234326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 762/1544] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:31.235363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.250788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 783/1545] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 17 11:38:31.253716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046b5d-e9a8-4173-9b5a-98bc030c7d96 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.254491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046b5d-e9a8-4173-9b5a-98bc030c7d96 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.257908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3347bf-47ff-4dcc-b817-d025f982efd1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.258604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046b5d-e9a8-4173-9b5a-98bc030c7d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.272605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d3639-6612-4a25-b7fd-90e33371abd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.274806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d3639-6612-4a25-b7fd-90e33371abd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.276691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d3639-6612-4a25-b7fd-90e33371abd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.277208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d3639-6612-4a25-b7fd-90e33371abd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.278198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d3639-6612-4a25-b7fd-90e33371abd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.289593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d3639-6612-4a25-b7fd-90e33371abd2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.313932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 763/1546] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:31.314773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-80046b5d-e9a8-4173-9b5a-98bc030c7d96 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.315520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa37e-472e-463b-801d-7ac921bedcd7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.315812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa37e-472e-463b-801d-7ac921bedcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.316088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa37e-472e-463b-801d-7ac921bedcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.317191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-80046b5d-e9a8-4173-9b5a-98bc030c7d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.331731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 784/1547] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.333441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e260f2-e44a-4891-8b21-c3cc5928d35b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.335654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e260f2-e44a-4891-8b21-c3cc5928d35b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.337634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e260f2-e44a-4891-8b21-c3cc5928d35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.353337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ac7aa37e-472e-463b-801d-7ac921bedcd7 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.355017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ac7aa37e-472e-463b-801d-7ac921bedcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.363982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 764/1548] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.364468 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f5e260f2-e44a-4891-8b21-c3cc5928d35b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.365705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f5e260f2-e44a-4891-8b21-c3cc5928d35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.366243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53c34-60dd-40c6-a971-31dd5d2bee79 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.366742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53c34-60dd-40c6-a971-31dd5d2bee79 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.367221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c53c34-60dd-40c6-a971-31dd5d2bee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.377514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 785/1549] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.381850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e260f2-e44a-4891-8b21-c3cc5928d35b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.389680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-58c53c34-60dd-40c6-a971-31dd5d2bee79 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.391192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-58c53c34-60dd-40c6-a971-31dd5d2bee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.398671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e260f2-e44a-4891-8b21-c3cc5928d35b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.404699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 765/1550] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.407109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af99d-02d0-42bd-a0d6-8bacb625ae03 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.407405 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af99d-02d0-42bd-a0d6-8bacb625ae03 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.407689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0af99d-02d0-42bd-a0d6-8bacb625ae03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.410778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a37b1d-43b2-4883-9092-7f412eec604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.411971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37b1d-43b2-4883-9092-7f412eec604e admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.412579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37b1d-43b2-4883-9092-7f412eec604e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.413153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37b1d-43b2-4883-9092-7f412eec604e admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.414206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a37b1d-43b2-4883-9092-7f412eec604e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.415842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a37b1d-43b2-4883-9092-7f412eec604e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.427764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 786/1551] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:31.428230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1d0af99d-02d0-42bd-a0d6-8bacb625ae03 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.429275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010cdcc-2e19-4829-840d-82daa9c00067 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.429523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010cdcc-2e19-4829-840d-82daa9c00067 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.429808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010cdcc-2e19-4829-840d-82daa9c00067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.430305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1d0af99d-02d0-42bd-a0d6-8bacb625ae03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.446091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 766/1552] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.447411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0af99d-02d0-42bd-a0d6-8bacb625ae03 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.457510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c010cdcc-2e19-4829-840d-82daa9c00067 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.458899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c010cdcc-2e19-4829-840d-82daa9c00067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.459112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0af99d-02d0-42bd-a0d6-8bacb625ae03 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.463862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d826-6a95-43ab-bdf0-ea28db41883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.464536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d826-6a95-43ab-bdf0-ea28db41883d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.464907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d826-6a95-43ab-bdf0-ea28db41883d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.465131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d826-6a95-43ab-bdf0-ea28db41883d admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.466410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d826-6a95-43ab-bdf0-ea28db41883d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:31.466723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d826-6a95-43ab-bdf0-ea28db41883d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.467814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 787/1553] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.468965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113618-a037-4878-9b5b-996ffa0ecc88 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.469319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113618-a037-4878-9b5b-996ffa0ecc88 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.469678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e113618-a037-4878-9b5b-996ffa0ecc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.470643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d826-6a95-43ab-bdf0-ea28db41883d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.487553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 767/1554] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:31.488228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8e113618-a037-4878-9b5b-996ffa0ecc88 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.489460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8e113618-a037-4878-9b5b-996ffa0ecc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.489869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bad53-2685-4527-999e-eb10c9509eb9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.490483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bad53-2685-4527-999e-eb10c9509eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.490957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bad53-2685-4527-999e-eb10c9509eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.498449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 788/1555] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.499753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537c4a-5213-4602-87a0-7207f0565104 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.500050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537c4a-5213-4602-87a0-7207f0565104 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.500331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea537c4a-5213-4602-87a0-7207f0565104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.511145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-841bad53-2685-4527-999e-eb10c9509eb9 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.512338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-841bad53-2685-4527-999e-eb10c9509eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.518590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ea537c4a-5213-4602-87a0-7207f0565104 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.519761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ea537c4a-5213-4602-87a0-7207f0565104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.520409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 768/1556] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.520996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bad53-2685-4527-999e-eb10c9509eb9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.532564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 789/1557] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.533486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537c4a-5213-4602-87a0-7207f0565104 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.534523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bad53-2685-4527-999e-eb10c9509eb9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.539495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2d5da-0e95-4e22-a74f-82f0ee7bdfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.540381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2d5da-0e95-4e22-a74f-82f0ee7bdfac admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.540666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2d5da-0e95-4e22-a74f-82f0ee7bdfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.540941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2d5da-0e95-4e22-a74f-82f0ee7bdfac admin admin] PATH_INFO: `/v3/projects/a957d18da3074f93a64089c5926549e0/users/311061eeaed24854aaa3cfaccc3a4af0/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.543254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea537c4a-5213-4602-87a0-7207f0565104 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.547761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.548410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.548693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.549111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.549852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:31.550195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.551198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:38:31.551198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 17 11:38:31.551198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: Jun 17 11:38:31.552066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e052c-0138-4d05-b814-5b14c11cbb70 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.555494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2d5da-0e95-4e22-a74f-82f0ee7bdfac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a957d18da3074f93a64089c5926549e0, user_id=311061eeaed24854aaa3cfaccc3a4af0, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.556520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-02f2d5da-0e95-4e22-a74f-82f0ee7bdfac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:38:31.556520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:38:31.556520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:38:31.559005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2d5da-0e95-4e22-a74f-82f0ee7bdfac admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.564969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 790/1558] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:31.566350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc8fe-17c5-4aa3-9ba5-b4786d0875c5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.566637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc8fe-17c5-4aa3-9ba5-b4786d0875c5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.566912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc8fe-17c5-4aa3-9ba5-b4786d0875c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.573818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 769/1559] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:31 2022] PUT /identity/v3/projects/a957d18da3074f93a64089c5926549e0/users/311061eeaed24854aaa3cfaccc3a4af0/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:31.575262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf744a-b811-4b37-84f6-b796a4663cee None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.575559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf744a-b811-4b37-84f6-b796a4663cee None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.575837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf744a-b811-4b37-84f6-b796a4663cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.590766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e79bc8fe-17c5-4aa3-9ba5-b4786d0875c5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.591967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e79bc8fe-17c5-4aa3-9ba5-b4786d0875c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.593626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-86cf744a-b811-4b37-84f6-b796a4663cee None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.595073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-86cf744a-b811-4b37-84f6-b796a4663cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.649535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:31.656383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 791/1560] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.657341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc8fe-17c5-4aa3-9ba5-b4786d0875c5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.662898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 770/1561] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.663571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf744a-b811-4b37-84f6-b796a4663cee None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.667353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc8fe-17c5-4aa3-9ba5-b4786d0875c5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.671964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.672620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.672902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.673176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.673651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf744a-b811-4b37-84f6-b796a4663cee None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.673886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.675794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.678265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63052fc-b7a5-4347-ad08-e7ae18f03cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.678927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63052fc-b7a5-4347-ad08-e7ae18f03cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.679212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63052fc-b7a5-4347-ad08-e7ae18f03cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.679539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63052fc-b7a5-4347-ad08-e7ae18f03cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.680240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63052fc-b7a5-4347-ad08-e7ae18f03cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:31.680547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63052fc-b7a5-4347-ad08-e7ae18f03cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.682743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63052fc-b7a5-4347-ad08-e7ae18f03cb3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.689692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:31.694728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 771/1562] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:31.696012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c87eb9-c067-448e-bee9-40a398397225 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.696305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c87eb9-c067-448e-bee9-40a398397225 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.696584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c87eb9-c067-448e-bee9-40a398397225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.707208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 792/1563] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:31.708171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.715907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e4c87eb9-c067-448e-bee9-40a398397225 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.717099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e4c87eb9-c067-448e-bee9-40a398397225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.771072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362064bd-c33c-4e60-968f-849562b9bccf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.775633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea377922-45ec-4b45-9ac8-ca295b971475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.776296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea377922-45ec-4b45-9ac8-ca295b971475 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.776700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea377922-45ec-4b45-9ac8-ca295b971475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.776985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea377922-45ec-4b45-9ac8-ca295b971475 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.777714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea377922-45ec-4b45-9ac8-ca295b971475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:31.778055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea377922-45ec-4b45-9ac8-ca295b971475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.778768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 772/1564] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.779550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c87eb9-c067-448e-bee9-40a398397225 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.782970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea377922-45ec-4b45-9ac8-ca295b971475 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.789319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c87eb9-c067-448e-bee9-40a398397225 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.794816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.795469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.795755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.795861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 793/1565] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:31.796039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.796801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.796996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770db5d-228f-44cb-9d5b-6a6622421d4f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.797289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770db5d-228f-44cb-9d5b-6a6622421d4f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.797565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770db5d-228f-44cb-9d5b-6a6622421d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.797825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:38:31.797825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:38:31.797825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:38:31.799058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.824247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f770db5d-228f-44cb-9d5b-6a6622421d4f None None] MFA Rules not processed for user `311061eeaed24854aaa3cfaccc3a4af0`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.825445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f770db5d-228f-44cb-9d5b-6a6622421d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.828748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 773/1566] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:31.828748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.888946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9372b18-ec9d-429c-9239-295fa46a9732 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.891309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 794/1567] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.893575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.894270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.894554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.894827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.895566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:31.895871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.897747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.903279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada0fb4-8923-43ae-b2a3-4b2b6a0b019b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.903889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada0fb4-8923-43ae-b2a3-4b2b6a0b019b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.904180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada0fb4-8923-43ae-b2a3-4b2b6a0b019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.918753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 774/1568] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:31.918753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.925827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8ada0fb4-8923-43ae-b2a3-4b2b6a0b019b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:31.927040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8ada0fb4-8923-43ae-b2a3-4b2b6a0b019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:31.932579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a6bab-5f4e-49e0-9524-b28918ac600b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.936753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 795/1569] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:31.937708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.938447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.938881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.939025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.939686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.941685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.941943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada0fb4-8923-43ae-b2a3-4b2b6a0b019b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.951639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada0fb4-8923-43ae-b2a3-4b2b6a0b019b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:31.958290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:31.959007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:31.959328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:31.959562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:31.960312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:31.961333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:38:31.961333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 17 11:38:31.961333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: Jun 17 11:38:31.962680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:31.983679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 775/1570] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:31.984824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:31.998233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 796/1571] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/users => generated 293 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:31.999634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.010612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a3f9a-3e10-4f71-8769-48777dbedebe admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.023112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:32.052855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46be60-ff54-4898-8d6b-5b142569836d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.057792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.058482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.058770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.059044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.059691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:32.059831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:32.060070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.061943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.069782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.070645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.070931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.071209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.071896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.073802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 776/1572] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:32.073954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.074482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.091589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7465c1-23f7-474f-8bff-c466bd8bcf3c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.096356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170d187-64f5-4a14-98ad-34d490477800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.097028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170d187-64f5-4a14-98ad-34d490477800 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.097317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170d187-64f5-4a14-98ad-34d490477800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.097599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170d187-64f5-4a14-98ad-34d490477800 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.098332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170d187-64f5-4a14-98ad-34d490477800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.102980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 797/1573] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:31 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.103137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170d187-64f5-4a14-98ad-34d490477800 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.103255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.123583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 777/1574] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/users => generated 282 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.123583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e331d-88c1-4a92-a7a6-3041fc4a8ca6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.123583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e331d-88c1-4a92-a7a6-3041fc4a8ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.123583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e331d-88c1-4a92-a7a6-3041fc4a8ca6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.124083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 778/1575] 10.222.0.25 () {54 vars in 961 bytes} [Fri Jun 17 11:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:32.124687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e331d-88c1-4a92-a7a6-3041fc4a8ca6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.165470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54ef17-1ac1-46f6-83ec-daad1cafe31e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.171105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.171775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.172065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.172347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.173070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.175041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.189032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e331d-88c1-4a92-a7a6-3041fc4a8ca6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.193948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.194636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.194928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.195238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.195977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:32.196308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.197957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.200535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 798/1576] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.201431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.218930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 779/1577] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:32.218930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.225894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738b47-c48b-41db-bae3-25f55106768f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.267892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee2d61-4552-4a3e-9792-e8417df2d135 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.272635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.273315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.273607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.273991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.274601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.277091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.283172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.283848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.284379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.284824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.285553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.287171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.299340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 799/1579] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.299537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 780/1579] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.299999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.300782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.310213 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cec24-96f9-4866-8e3c-4d48aad0eb05 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.310425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647528a-deaf-490a-b002-02ae40af5a9b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.314692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.315056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbe0cf-e1f8-4a35-85e9-0c62f2d9877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.315343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.315625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.315730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbe0cf-e1f8-4a35-85e9-0c62f2d9877c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.315900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.315996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbe0cf-e1f8-4a35-85e9-0c62f2d9877c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.316237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbe0cf-e1f8-4a35-85e9-0c62f2d9877c admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.316582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.316929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbe0cf-e1f8-4a35-85e9-0c62f2d9877c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.318526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.318912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbe0cf-e1f8-4a35-85e9-0c62f2d9877c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.339566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 781/1580] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/users => generated 296 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.340214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.349619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 800/1581] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.349898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da0ecd-eb75-407c-b47d-a67ca6c75215 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.351159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6512b0c-3c36-4b6e-92ce-0fd6cff3d464 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.351738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6512b0c-3c36-4b6e-92ce-0fd6cff3d464 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.351958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6512b0c-3c36-4b6e-92ce-0fd6cff3d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.368884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:32.369108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b6512b0c-3c36-4b6e-92ce-0fd6cff3d464 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:32.370449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b6512b0c-3c36-4b6e-92ce-0fd6cff3d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:32.398024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:32.408928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.409901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.410182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.410456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.411209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.412918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.423508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 782/1582] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.427240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.436601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744c70b-74d1-45a9-8725-0103244f225e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.440798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 801/1583] 10.222.0.25 () {58 vars in 1058 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:32.441099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.441749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.441989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6512b0c-3c36-4b6e-92ce-0fd6cff3d464 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.442110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.442330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.443026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.444940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.452745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6512b0c-3c36-4b6e-92ce-0fd6cff3d464 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.458056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.458766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.459096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.459399 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.460087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.462018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.466015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 783/1584] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/users => generated 302 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.466824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.476045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a565d3-663a-4bc5-bd0a-d239d81c468a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.480573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.481218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.481533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.481816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.482534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.484228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.494725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 802/1585] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.494725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.501287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 784/1586] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.501287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.508521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09c0e4-2752-402b-b608-251e0cb1d0f1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.556265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4f62c-125c-436e-8056-edc974b74f01 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.560852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.561524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.561812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.562382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.563107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.564549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.564718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.565352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.565641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.566066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.566804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.568395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.575071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 803/1587] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.575928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.579202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 785/1588] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.579904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.586937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324f2c2-9738-447f-b2d3-a517bc7430b8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.589895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12763537-6d29-417f-af1e-f39c4adebcae admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.592108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.592880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.593163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.593447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.594281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.594816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.595527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.595899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.596183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.596278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.597063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.598736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.609806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 786/1589] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.610563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.618916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 804/1590] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/users => generated 287 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.618916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.626332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71499cf-14cb-4c37-be12-a2b49f8e7017 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.631274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.631934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.632222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.632494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.633247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.634852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.646673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 805/1591] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.650634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.660758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce82175-1df3-4fe0-8cef-7af7875f1c7a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.665637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.666596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.666989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.667264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.667947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.669856 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.690449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 806/1592] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/users => generated 303 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:32.691498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.692842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2d3f4-5317-4a70-8c82-6f02ed769400 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.698985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.699644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.699933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.700208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.703267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153e7b6-c356-4db1-bcbe-09c18080eca7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.707905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.708769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.709053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.709325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] PATH_INFO: `/v3/projects/03b64e74629c4d6b9c5ef82247509762/users/c2c930cfab47464fb784c98f58abbb2c/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.709689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:32.725309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b64e74629c4d6b9c5ef82247509762, user_id=c2c930cfab47464fb784c98f58abbb2c, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.728041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.742085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 807/1593] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:32 2022] PUT /identity/v3/projects/03b64e74629c4d6b9c5ef82247509762/users/c2c930cfab47464fb784c98f58abbb2c/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:32.742900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.761273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.762935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.772444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 787/1594] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:32.773240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.775401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:32.785535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c512d1-4ede-4934-a997-2de54971df15 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.813191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383399-6fd1-44c4-ab6f-e39bbdad7f0f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.818179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.818981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.819371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.819754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.820590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.822312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.838127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 808/1595] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.838127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.849175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2f676-faf6-4582-ba6c-75dfee676fb1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.852030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.852809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.852974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.853264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.854037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.855584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.855822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.856717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.857224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.857579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] PATH_INFO: `/v3/projects/d82c5dacbdea44588de60dcf520afdf1/users/d9db3bea46a34ea69f73e0fb355480b8/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.869822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 788/1596] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.869822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.877576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d82c5dacbdea44588de60dcf520afdf1, user_id=d9db3bea46a34ea69f73e0fb355480b8, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.879836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed432f8-619f-4ed3-b093-b9877d7805ab admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.880458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.886261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.887048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.887452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.887867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.888653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.890481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.905227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 809/1597] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:32 2022] PUT /identity/v3/projects/d82c5dacbdea44588de60dcf520afdf1/users/d9db3bea46a34ea69f73e0fb355480b8/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:32.905227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.905820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 789/1598] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.905820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.912686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95923c3-15bb-4699-8157-8f3113a633c9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.973806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabd8b0-ebed-48a4-8dc1-1d86bc34ffd2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:32.976330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16ce5b-fed5-4267-bce9-f72c0305c7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.977077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ce5b-fed5-4267-bce9-f72c0305c7ac admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.977426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ce5b-fed5-4267-bce9-f72c0305c7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.977787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16ce5b-fed5-4267-bce9-f72c0305c7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.978784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ce5b-fed5-4267-bce9-f72c0305c7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.978905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:32.979596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.979960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.980308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:32.980781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16ce5b-fed5-4267-bce9-f72c0305c7ac admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.981169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:32.982853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:32.996028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 790/1599] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.996193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 810/1600] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:32.996328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4ced-ab39-4997-848f-df0c9cbb1eed None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:32.996328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4ced-ab39-4997-848f-df0c9cbb1eed None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:32.996511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:32.996603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4ced-ab39-4997-848f-df0c9cbb1eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.005862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a7362-ba49-4f4d-a907-c1227b68f9fa admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.011439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.012395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.012749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.013100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] PATH_INFO: `/v3/projects/8a16c8fcc2f046fd9b1309ea822eb5b5/users/6e2a2c6831ba4d13aeb6e550f8eff9a6/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.014977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8cbb4ced-ab39-4997-848f-df0c9cbb1eed None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.016202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8cbb4ced-ab39-4997-848f-df0c9cbb1eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.024076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 791/1601] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:33.024839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4ced-ab39-4997-848f-df0c9cbb1eed None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.030534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a16c8fcc2f046fd9b1309ea822eb5b5, user_id=6e2a2c6831ba4d13aeb6e550f8eff9a6, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.033283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.034902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4ced-ab39-4997-848f-df0c9cbb1eed None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.039765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c97c36-c5c6-4f5d-9f3d-fc474b919166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.040666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c97c36-c5c6-4f5d-9f3d-fc474b919166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.040954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c97c36-c5c6-4f5d-9f3d-fc474b919166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.041297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c97c36-c5c6-4f5d-9f3d-fc474b919166 admin admin] PATH_INFO: `/v3/projects/b54e176d82d943af98e5f81ee153caab/users/9e209f3667f54c2b9bc6eab13010296c/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.046765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 811/1602] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:32 2022] PUT /identity/v3/projects/8a16c8fcc2f046fd9b1309ea822eb5b5/users/6e2a2c6831ba4d13aeb6e550f8eff9a6/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:33.047686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.061637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c97c36-c5c6-4f5d-9f3d-fc474b919166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b54e176d82d943af98e5f81ee153caab, user_id=9e209f3667f54c2b9bc6eab13010296c, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.064510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c97c36-c5c6-4f5d-9f3d-fc474b919166 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.077546 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 792/1603] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:33 2022] PUT /identity/v3/projects/b54e176d82d943af98e5f81ee153caab/users/9e209f3667f54c2b9bc6eab13010296c/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:33.078871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6027c2-77a0-4fc0-8560-572c6c3bb48b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.079157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6027c2-77a0-4fc0-8560-572c6c3bb48b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.079428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6027c2-77a0-4fc0-8560-572c6c3bb48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.084909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:33.107036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ea6027c2-77a0-4fc0-8560-572c6c3bb48b None None] MFA Rules not processed for user `d9db3bea46a34ea69f73e0fb355480b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.108245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ea6027c2-77a0-4fc0-8560-572c6c3bb48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.121493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb88514-4fdf-4546-a8a9-324379b370e5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.133601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:33.174360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 793/1604] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:33.175577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6027c2-77a0-4fc0-8560-572c6c3bb48b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.177491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.178196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.178476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.178749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.179472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.181066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.186486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6027c2-77a0-4fc0-8560-572c6c3bb48b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.191296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6360e8-af55-4a2a-9aa9-44069915ea1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.191737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 812/1605] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:33.191957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6360e8-af55-4a2a-9aa9-44069915ea1f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.192306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6360e8-af55-4a2a-9aa9-44069915ea1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.192413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.192505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6360e8-af55-4a2a-9aa9-44069915ea1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.193374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6360e8-af55-4a2a-9aa9-44069915ea1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.195168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6360e8-af55-4a2a-9aa9-44069915ea1f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.201475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d06c2-cbb6-4d83-851e-2038770f3c0c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.205741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 794/1606] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:33.205961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf3e49-7440-4978-bf81-94b5a465a913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.206881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf3e49-7440-4978-bf81-94b5a465a913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.207166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf3e49-7440-4978-bf81-94b5a465a913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.207275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37411f-46ac-4c9a-8a8c-a285cee1a3a4 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.207439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf3e49-7440-4978-bf81-94b5a465a913 admin admin] PATH_INFO: `/v3/projects/4bf1279fc40c41dd8f652e1d80bb0314/users/4a66f80e4c814767bca999f00e95511e/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.207686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37411f-46ac-4c9a-8a8c-a285cee1a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.208014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37411f-46ac-4c9a-8a8c-a285cee1a3a4 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.209443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 795/1607] 10.222.0.25 () {54 vars in 912 bytes} [Fri Jun 17 11:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:33.214486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37411f-46ac-4c9a-8a8c-a285cee1a3a4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.222138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf3e49-7440-4978-bf81-94b5a465a913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf1279fc40c41dd8f652e1d80bb0314, user_id=4a66f80e4c814767bca999f00e95511e, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.224844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf3e49-7440-4978-bf81-94b5a465a913 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.238199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 813/1608] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:33 2022] PUT /identity/v3/projects/4bf1279fc40c41dd8f652e1d80bb0314/users/4a66f80e4c814767bca999f00e95511e/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:33.242929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e909a-ca3e-4843-93a0-6093583ea290 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.243219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e909a-ca3e-4843-93a0-6093583ea290 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.244091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e909a-ca3e-4843-93a0-6093583ea290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.262079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-548e909a-ca3e-4843-93a0-6093583ea290 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.263291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-548e909a-ca3e-4843-93a0-6093583ea290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.271558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37411f-46ac-4c9a-8a8c-a285cee1a3a4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.321724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 814/1609] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:33.325178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819193b1-f317-45d0-94fa-6a047189b836 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.325178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819193b1-f317-45d0-94fa-6a047189b836 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.325382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.325620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819193b1-f317-45d0-94fa-6a047189b836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.325723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.325723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.325723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.334487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.335923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.341676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-819193b1-f317-45d0-94fa-6a047189b836 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.342905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-819193b1-f317-45d0-94fa-6a047189b836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.345654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 796/1610] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:33.346462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.351925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 815/1611] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:33.352773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819193b1-f317-45d0-94fa-6a047189b836 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.356692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daf075-cf99-4ff4-85c9-e8f38fc690ef service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.362796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.363807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819193b1-f317-45d0-94fa-6a047189b836 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.363947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.363947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.364261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.365028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.366953 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.368464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4667bf0-fe4c-4e08-9672-ec7ebec17680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.369532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4667bf0-fe4c-4e08-9672-ec7ebec17680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.370672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4667bf0-fe4c-4e08-9672-ec7ebec17680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.371019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4667bf0-fe4c-4e08-9672-ec7ebec17680 admin admin] PATH_INFO: `/v3/projects/fad50b8414f246428a34e0465810ab97/users/3a0f25c7ce194ddbb21d1c8b97d03c8c/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.388305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4667bf0-fe4c-4e08-9672-ec7ebec17680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad50b8414f246428a34e0465810ab97, user_id=3a0f25c7ce194ddbb21d1c8b97d03c8c, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.391109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4667bf0-fe4c-4e08-9672-ec7ebec17680 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.397453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 797/1612] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:33.400815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.406907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 816/1613] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:33 2022] PUT /identity/v3/projects/fad50b8414f246428a34e0465810ab97/users/3a0f25c7ce194ddbb21d1c8b97d03c8c/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:33.408175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64831e95-3872-470a-a2ed-551d6d7902f8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.408463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64831e95-3872-470a-a2ed-551d6d7902f8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.408738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64831e95-3872-470a-a2ed-551d6d7902f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.424144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:33.429120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-64831e95-3872-470a-a2ed-551d6d7902f8 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.430503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-64831e95-3872-470a-a2ed-551d6d7902f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.474424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e824e-73d9-4113-87ff-796f777db159 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.493880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:33.517989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 817/1614] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:33.520321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78b30-3e32-4976-b17f-53916c8ed627 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.520832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78b30-3e32-4976-b17f-53916c8ed627 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.522240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78b30-3e32-4976-b17f-53916c8ed627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.543647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d1b78b30-3e32-4976-b17f-53916c8ed627 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.545343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1b78b30-3e32-4976-b17f-53916c8ed627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.545555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.546514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.546853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.547123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] PATH_INFO: `/v3/projects/ea5f1c52ff18447e9d596cc9e677ca70/users/20fdb7f784d64fcf9a5eaef75c0642a4/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.563405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 818/1615] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:33.564262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b78b30-3e32-4976-b17f-53916c8ed627 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.574066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5f1c52ff18447e9d596cc9e677ca70, user_id=20fdb7f784d64fcf9a5eaef75c0642a4, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.577751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.591634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 798/1616] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:33 2022] PUT /identity/v3/projects/ea5f1c52ff18447e9d596cc9e677ca70/users/20fdb7f784d64fcf9a5eaef75c0642a4/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:33.602183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.604284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.639693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b78b30-3e32-4976-b17f-53916c8ed627 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.675193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceffed1-ac21-45a0-b656-dec95bc37514 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.679455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.679983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.680929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.681284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.705686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6873fa-6525-44b9-a062-7d338e6b1820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.706641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6873fa-6525-44b9-a062-7d338e6b1820 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.706908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6873fa-6525-44b9-a062-7d338e6b1820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.707258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6873fa-6525-44b9-a062-7d338e6b1820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.759322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.761563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.767528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:33.778177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 799/1617] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:33.779283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.783275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6873fa-6525-44b9-a062-7d338e6b1820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.785118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6873fa-6525-44b9-a062-7d338e6b1820 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.792204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829825-4d2f-4402-8f09-008ca36798a7 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.797271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 819/1618] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:33.798319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfa61a-30f6-4077-af6d-12bb63d125c8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.798613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfa61a-30f6-4077-af6d-12bb63d125c8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.798892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfa61a-30f6-4077-af6d-12bb63d125c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.806391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.807747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.808127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.808621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.811236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.813292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.820491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a3cfa61a-30f6-4077-af6d-12bb63d125c8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.821833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a3cfa61a-30f6-4077-af6d-12bb63d125c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:33.832908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 820/1619] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:33.833778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfa61a-30f6-4077-af6d-12bb63d125c8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.842874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 800/1620] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/users => generated 289 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:33.843811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:33.858065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711acc77-79e1-4f37-be15-625238f19c60 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.863077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79e00d-57f7-4be8-8a4b-34290059b71c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.863832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79e00d-57f7-4be8-8a4b-34290059b71c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.864198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79e00d-57f7-4be8-8a4b-34290059b71c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.864570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79e00d-57f7-4be8-8a4b-34290059b71c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.878998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79e00d-57f7-4be8-8a4b-34290059b71c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:33.881169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79e00d-57f7-4be8-8a4b-34290059b71c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:33.894004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 801/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:33.895980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f9dd-0f05-463a-9765-e63fc465588c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.895980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f9dd-0f05-463a-9765-e63fc465588c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.896225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2f9dd-0f05-463a-9765-e63fc465588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.898268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:33.911763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfa61a-30f6-4077-af6d-12bb63d125c8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:33.916847 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:33.918061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:33.918354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:33.918640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:33.926997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-def2f9dd-0f05-463a-9765-e63fc465588c None None] MFA Rules not processed for user `20fdb7f784d64fcf9a5eaef75c0642a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:33.928317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-def2f9dd-0f05-463a-9765-e63fc465588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:34.006760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 802/1622] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:34.008080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2f9dd-0f05-463a-9765-e63fc465588c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.017474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.019142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.021362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2f9dd-0f05-463a-9765-e63fc465588c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.028384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.029450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.029580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.030003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.030731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.032734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.042260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 821/1623] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:34.043706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.055267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4647e7-1367-4c6d-bf14-38bec09fd545 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.061183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 803/1624] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:34.062702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.065126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.066241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.066542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.066863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.079542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.081049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.090088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8eb4fd-a68e-401f-9176-3cf73600b9a8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.093177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 822/1625] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:34.094109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.099688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.100630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.101110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.101661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.102723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.105896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.112269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fb92c-ce34-4741-bcba-9ff2983650c9 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.118474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.120347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.120694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.121038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] PATH_INFO: `/v3/projects/76d57ff0aad740d49fc347486e0c5b22/users/0059f4a005af4048bd3fdc6b39dd27e6/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.142561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d57ff0aad740d49fc347486e0c5b22, user_id=0059f4a005af4048bd3fdc6b39dd27e6, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.146485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.172513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 823/1626] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:34 2022] PUT /identity/v3/projects/76d57ff0aad740d49fc347486e0c5b22/users/0059f4a005af4048bd3fdc6b39dd27e6/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:34.175518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.177200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 804/1627] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:34 2022] POST /identity/v3/projects => generated 405 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:34.181580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.267463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f77697-17a1-43c1-ab54-3859cde649a0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.328414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b79689-ba12-4dde-8d52-5dc883f44c46 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.336814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:34.343764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.344464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.344752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.345027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.345730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.347702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.360828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.363880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.364230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.365043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.369153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.375493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.393162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 805/1628] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:34 2022] POST /identity/v3/users => generated 309 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:34.394728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.404023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 824/1629] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:34.411755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f831a7-0806-43dc-84ed-05dc56d14261 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.413969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.420357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.421322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.421565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.422828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.423646 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.425925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.434889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131c967-ef06-4dfe-a926-771aef0e39c9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.440262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 806/1630] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:34.444425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.447827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f9f06-024a-4ef4-b63f-2ce5786b1b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.449177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9f06-024a-4ef4-b63f-2ce5786b1b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.449542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9f06-024a-4ef4-b63f-2ce5786b1b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.449777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9f06-024a-4ef4-b63f-2ce5786b1b18 admin admin] PATH_INFO: `/v3/projects/76d57ff0aad740d49fc347486e0c5b22/users/0059f4a005af4048bd3fdc6b39dd27e6/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.452032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f9f06-024a-4ef4-b63f-2ce5786b1b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d57ff0aad740d49fc347486e0c5b22, user_id=0059f4a005af4048bd3fdc6b39dd27e6, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.453080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:34.455850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f9f06-024a-4ef4-b63f-2ce5786b1b18 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.471377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6f598-59f1-4073-935b-7485e6507be4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.471582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 825/1631] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:34 2022] PUT /identity/v3/projects/76d57ff0aad740d49fc347486e0c5b22/users/0059f4a005af4048bd3fdc6b39dd27e6/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:34.483402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bc5c-d7ed-4542-9093-c2865809dd7a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.484013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bc5c-d7ed-4542-9093-c2865809dd7a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.484447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bc5c-d7ed-4542-9093-c2865809dd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.561021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.562594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.562594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.562764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] PATH_INFO: `/v3/projects/7371b36d70464ffb9d8c219d337f974a/users/15297d7160a5429683faf24b3a3824cb/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.566332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c280bc5c-d7ed-4542-9093-c2865809dd7a None None] MFA Rules not processed for user `0059f4a005af4048bd3fdc6b39dd27e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:34.570752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c280bc5c-d7ed-4542-9093-c2865809dd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:34.581480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7371b36d70464ffb9d8c219d337f974a, user_id=15297d7160a5429683faf24b3a3824cb, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.585529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.619689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 807/1632] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:34 2022] PUT /identity/v3/projects/7371b36d70464ffb9d8c219d337f974a/users/15297d7160a5429683faf24b3a3824cb/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:34.620993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.773059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c169-dd43-44eb-b8a4-8ae1e67cb5d5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.777987 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 826/1633] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:38:34.783343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.784393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.785173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.785173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.785800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.787897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.792909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280bc5c-d7ed-4542-9093-c2865809dd7a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.823830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 808/1634] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:34.829024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:34.852259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b906d06-4e32-4179-9dde-89352361129e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.866802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d059d5-e833-4989-b35b-6f4cfd6fa58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:34.868718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d059d5-e833-4989-b35b-6f4cfd6fa58e admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.869007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d059d5-e833-4989-b35b-6f4cfd6fa58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.869319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d059d5-e833-4989-b35b-6f4cfd6fa58e admin admin] PATH_INFO: `/v3/projects/7371b36d70464ffb9d8c219d337f974a/users/15297d7160a5429683faf24b3a3824cb/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.874468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d059d5-e833-4989-b35b-6f4cfd6fa58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7371b36d70464ffb9d8c219d337f974a, user_id=15297d7160a5429683faf24b3a3824cb, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:34.879779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d059d5-e833-4989-b35b-6f4cfd6fa58e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:34.912844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 809/1635] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:34 2022] PUT /identity/v3/projects/7371b36d70464ffb9d8c219d337f974a/users/15297d7160a5429683faf24b3a3824cb/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:34.925370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bddee-0a15-48d3-b9f1-e0458b2cb445 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:34.925662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bddee-0a15-48d3-b9f1-e0458b2cb445 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:34.925996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bddee-0a15-48d3-b9f1-e0458b2cb445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:34.929356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280bc5c-d7ed-4542-9093-c2865809dd7a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:34.962284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-067bddee-0a15-48d3-b9f1-e0458b2cb445 None None] MFA Rules not processed for user `15297d7160a5429683faf24b3a3824cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:34.964165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-067bddee-0a15-48d3-b9f1-e0458b2cb445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:35.048432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9115802-769f-486c-bd1f-c92eab82778c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:35.050606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9115802-769f-486c-bd1f-c92eab82778c service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:35.050901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9115802-769f-486c-bd1f-c92eab82778c service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:35.051219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9115802-769f-486c-bd1f-c92eab82778c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:35.113226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:35.129865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:35.134941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 810/1636] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:34 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:35.171673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9115802-769f-486c-bd1f-c92eab82778c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:35.173212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9115802-769f-486c-bd1f-c92eab82778c service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:35.189476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bddee-0a15-48d3-b9f1-e0458b2cb445 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:35.205920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bddee-0a15-48d3-b9f1-e0458b2cb445 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:35.216575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 827/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:35.218014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d24df1-8874-4b10-ac5d-2d32470813da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:35.218634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d24df1-8874-4b10-ac5d-2d32470813da service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:35.218926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d24df1-8874-4b10-ac5d-2d32470813da service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:35.219243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d24df1-8874-4b10-ac5d-2d32470813da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:35.241575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d24df1-8874-4b10-ac5d-2d32470813da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:35.244672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d24df1-8874-4b10-ac5d-2d32470813da service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:35.269709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 811/1638] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:35 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:38:35.677940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3aaa-a734-4fb5-ac87-97323a805094 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:35.678624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3aaa-a734-4fb5-ac87-97323a805094 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:35.678897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b3aaa-a734-4fb5-ac87-97323a805094 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:35.681563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 828/1639] 10.222.0.25 () {54 vars in 961 bytes} [Fri Jun 17 11:38:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:35.696145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163d0ed-05f6-444a-bbab-9a31621d9703 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:35.700142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163d0ed-05f6-444a-bbab-9a31621d9703 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:35.700457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163d0ed-05f6-444a-bbab-9a31621d9703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:35.744990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1163d0ed-05f6-444a-bbab-9a31621d9703 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:35.747552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1163d0ed-05f6-444a-bbab-9a31621d9703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:35.750540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3aaa-a734-4fb5-ac87-97323a805094 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:35.907271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b3aaa-a734-4fb5-ac87-97323a805094 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:35.913379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:35.914401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:35.914795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:35.915143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] PATH_INFO: `/v3/users/20fdb7f784d64fcf9a5eaef75c0642a4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:35.916893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fdb7f784d64fcf9a5eaef75c0642a4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:35.919757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:35.973048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 812/1640] 10.222.0.25 () {58 vars in 1058 bytes} [Fri Jun 17 11:38:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:35.975084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e401-f977-47b1-83d5-6e5b7ec10400 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:35.975384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e401-f977-47b1-83d5-6e5b7ec10400 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:35.975675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e401-f977-47b1-83d5-6e5b7ec10400 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:35.984867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 813/1641] 10.222.0.25 () {54 vars in 912 bytes} [Fri Jun 17 11:38:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:35.986026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9186282-e4c0-4f1b-8b60-84daf88044f4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:35.986314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9186282-e4c0-4f1b-8b60-84daf88044f4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:35.993320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9186282-e4c0-4f1b-8b60-84daf88044f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:36.020447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a9186282-e4c0-4f1b-8b60-84daf88044f4 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:36.022009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a9186282-e4c0-4f1b-8b60-84daf88044f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:36.047168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 814/1642] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:38:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:36.081665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fdb7f784d64fcf9a5eaef75c0642a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:36.084239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9186282-e4c0-4f1b-8b60-84daf88044f4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:36.101988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fdb7f784d64fcf9a5eaef75c0642a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:36.116787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fdb7f784d64fcf9a5eaef75c0642a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:36.116787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fdb7f784d64fcf9a5eaef75c0642a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:36.139311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d6157880-f9e2-455a-89da-ced7c313dc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fdb7f784d64fcf9a5eaef75c0642a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:36.141205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9186282-e4c0-4f1b-8b60-84daf88044f4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:36.164581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:36.165690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:36.168932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 829/1643] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:35 2022] DELETE /identity/v3/users/20fdb7f784d64fcf9a5eaef75c0642a4 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:36.169303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:36.169614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:36.176951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:36.177552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:36.179304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:36.180074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:36.246185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:36.249764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:36.263999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:36.284835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:36.311080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 815/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:38:36.677039 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:36.729947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 830/1645] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:36.814946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:36.821696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:36.853342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:36.856744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:36.886059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8776c-3380-4835-9baf-f86835dc4e56 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:36.892536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:36.894634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:36.895180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:36.895742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] PATH_INFO: `/v3/users/d9db3bea46a34ea69f73e0fb355480b8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:36.898328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9db3bea46a34ea69f73e0fb355480b8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:36.902824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:36.987133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b17518-648f-4297-842d-abcc7f0477b2 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:37.014216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9db3bea46a34ea69f73e0fb355480b8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:37.036474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9db3bea46a34ea69f73e0fb355480b8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:37.068797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32609e-8608-4df9-b1d5-fb42b8fb3780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:37.068797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32609e-8608-4df9-b1d5-fb42b8fb3780 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:37.068797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32609e-8608-4df9-b1d5-fb42b8fb3780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:37.068797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32609e-8608-4df9-b1d5-fb42b8fb3780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:37.075433 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9db3bea46a34ea69f73e0fb355480b8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:37.077142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9db3bea46a34ea69f73e0fb355480b8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:37.093420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32609e-8608-4df9-b1d5-fb42b8fb3780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:37.095432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32609e-8608-4df9-b1d5-fb42b8fb3780 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:37.109459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9db3bea46a34ea69f73e0fb355480b8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:37.114864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 816/1646] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:37.119780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 831/1647] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:36 2022] DELETE /identity/v3/users/d9db3bea46a34ea69f73e0fb355480b8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:37.132416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527029b5-b08b-45c3-8554-a09251234c58 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:37.132852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527029b5-b08b-45c3-8554-a09251234c58 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:37.133789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527029b5-b08b-45c3-8554-a09251234c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:37.188342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-527029b5-b08b-45c3-8554-a09251234c58 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:37.190719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-527029b5-b08b-45c3-8554-a09251234c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:37.202442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:37.406365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 817/1648] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:37.454026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:37.455319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:37.612545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64761b0-d335-44a1-9971-f94fbf2c6e4d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:37.627660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:37.630112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:37.631658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:37.632366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:37.657095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:37.659462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:37.678278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 832/1649] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:37.875218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31806553-8e6c-4e8a-9a85-7430269ae4d7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:37.875218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31806553-8e6c-4e8a-9a85-7430269ae4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:37.875984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31806553-8e6c-4e8a-9a85-7430269ae4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:37.959777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-31806553-8e6c-4e8a-9a85-7430269ae4d7 None None] MFA Rules not processed for user `4a66f80e4c814767bca999f00e95511e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:37.961056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-31806553-8e6c-4e8a-9a85-7430269ae4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:38.101772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 818/1650] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:38.106975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.109787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:38.131576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb44c38-d4fb-4db6-82fd-ee1991c35500 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.145010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.145712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.149699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.149699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.149699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.149699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.196137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 833/1651] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:38 2022] POST /identity/v3/projects => generated 403 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:38.200630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31806553-8e6c-4e8a-9a85-7430269ae4d7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.216128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.222472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:38.223145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31806553-8e6c-4e8a-9a85-7430269ae4d7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.371212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:38.404205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.405312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.405312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.406061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.406890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.408937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.419939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f3b30-0b5d-4022-8135-55e25f254cf1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.432928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.433666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.433996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.434270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] PATH_INFO: `/v3/projects/ea5f1c52ff18447e9d596cc9e677ca70` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.436059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea5f1c52ff18447e9d596cc9e677ca70)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.437105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:38:38.437105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 17 11:38:38.437105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: Jun 17 11:38:38.441158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 819/1652] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:38 2022] POST /identity/v3/users => generated 308 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:38:38.443044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.447850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.461739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd08d8-6676-4d6a-aff7-bf31a87259d6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.469261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.470655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.470952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.471234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.472027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.473674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.488979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 820/1653] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:38.490496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.504634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a56e34-b1e3-4e54-847f-23cd785e0d37 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.514004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.514983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.528425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.530530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] PATH_INFO: `/v3/projects/8b7b79252e75415e97e56b49392be147/users/eb171002aa064b4e9c2a21ead8a44229/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.562348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea5f1c52ff18447e9d596cc9e677ca70', 'request_id': 'req-5c11056a-81c1-4584-ba6a-146ddce9f074'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:38.569024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7b79252e75415e97e56b49392be147, user_id=eb171002aa064b4e9c2a21ead8a44229, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.574525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.580418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:38.583325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea5f1c52ff18447e9d596cc9e677ca70', 'request_id': 'req-5c11056a-81c1-4584-ba6a-146ddce9f074'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:38.597986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea5f1c52ff18447e9d596cc9e677ca70', 'request_id': 'req-5c11056a-81c1-4584-ba6a-146ddce9f074'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:38.624846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 821/1654] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:38 2022] PUT /identity/v3/projects/8b7b79252e75415e97e56b49392be147/users/eb171002aa064b4e9c2a21ead8a44229/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:38:38.633309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.657475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] The token cache is being invalidate because project ea5f1c52ff18447e9d596cc9e677ca70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:38.657671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:38.659387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:38.666216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 834/1655] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:38 2022] DELETE /identity/v3/projects/ea5f1c52ff18447e9d596cc9e677ca70 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:38.668153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.671703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:38.743773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc48ae0-89f3-4b4e-9c7e-7bebc76520ab admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.756172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.757302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.759503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.760711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.762165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.764921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.789588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 822/1656] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:38.797701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:38.805552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:38.818234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11056a-81c1-4584-ba6a-146ddce9f074 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.830478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.832766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.833057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.834476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] PATH_INFO: `/v3/users/15297d7160a5429683faf24b3a3824cb` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.838569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15297d7160a5429683faf24b3a3824cb)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.843355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711e22-9eac-4fb5-83d3-c7642747af0f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:38.845155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.856605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:38.857580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:38.857881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:38.858155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] PATH_INFO: `/v3/projects/8b7b79252e75415e97e56b49392be147/users/eb171002aa064b4e9c2a21ead8a44229/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:38.861009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7b79252e75415e97e56b49392be147, user_id=eb171002aa064b4e9c2a21ead8a44229, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:38.872764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:38.908098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 823/1657] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:38 2022] PUT /identity/v3/projects/8b7b79252e75415e97e56b49392be147/users/eb171002aa064b4e9c2a21ead8a44229/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:38.951465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15297d7160a5429683faf24b3a3824cb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:38.973188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15297d7160a5429683faf24b3a3824cb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:38.988892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15297d7160a5429683faf24b3a3824cb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.033713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15297d7160a5429683faf24b3a3824cb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.033713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:39.037066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15297d7160a5429683faf24b3a3824cb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.047175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 835/1658] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:38 2022] DELETE /identity/v3/users/15297d7160a5429683faf24b3a3824cb => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:39.418677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:39.420419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:39.565650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1c27c-e08f-4dc9-8986-cdd156519cee admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:39.682499 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:39.684814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:39.686144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:39.687811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] PATH_INFO: `/v3/projects/d82c5dacbdea44588de60dcf520afdf1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:39.690276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d82c5dacbdea44588de60dcf520afdf1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:39.692350 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:39.763288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:39.793215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd82c5dacbdea44588de60dcf520afdf1', 'request_id': 'req-aee4bef2-d233-424f-8986-607e7668f058'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.808122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd82c5dacbdea44588de60dcf520afdf1', 'request_id': 'req-aee4bef2-d233-424f-8986-607e7668f058'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.817079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd82c5dacbdea44588de60dcf520afdf1', 'request_id': 'req-aee4bef2-d233-424f-8986-607e7668f058'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.839294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] The token cache is being invalidate because project d82c5dacbdea44588de60dcf520afdf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:39.839754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.840750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:39.845833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 824/1659] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:39 2022] DELETE /identity/v3/projects/d82c5dacbdea44588de60dcf520afdf1 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:39.847415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:39.870406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67b7c-8f13-4b6b-a02a-ac29ae518650 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:39.983741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:39.985199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:39.985566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:39.985946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] PATH_INFO: `/v3/users/0059f4a005af4048bd3fdc6b39dd27e6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:39.988132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0059f4a005af4048bd3fdc6b39dd27e6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:39.990357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:40.094489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0059f4a005af4048bd3fdc6b39dd27e6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.108164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0059f4a005af4048bd3fdc6b39dd27e6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.119878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0059f4a005af4048bd3fdc6b39dd27e6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.120906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0059f4a005af4048bd3fdc6b39dd27e6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.141023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8b0335d9-edce-4651-af3f-052c3320c9b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0059f4a005af4048bd3fdc6b39dd27e6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.152620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 836/1660] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:39 2022] DELETE /identity/v3/users/0059f4a005af4048bd3fdc6b39dd27e6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:40.447843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:40.450040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:40.472288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4bef2-d233-424f-8986-607e7668f058 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:40.639277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:40.640312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:40.640753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:40.641243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] PATH_INFO: `/v3/projects/7371b36d70464ffb9d8c219d337f974a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:40.644544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7371b36d70464ffb9d8c219d337f974a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:40.655246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:40.771848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7371b36d70464ffb9d8c219d337f974a', 'request_id': 'req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.785917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7371b36d70464ffb9d8c219d337f974a', 'request_id': 'req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.814629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7371b36d70464ffb9d8c219d337f974a', 'request_id': 'req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.864374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] The token cache is being invalidate because project 7371b36d70464ffb9d8c219d337f974a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:40.864858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.865657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-79c97661-5071-4fe7-9ade-4d7bcfdf77f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:40.871848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 825/1661] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:40 2022] DELETE /identity/v3/projects/7371b36d70464ffb9d8c219d337f974a => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:40.872614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2912b99-74bd-4da8-a699-9b11b68c5a0c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:40.873037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2912b99-74bd-4da8-a699-9b11b68c5a0c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:40.873189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2912b99-74bd-4da8-a699-9b11b68c5a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:40.921633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e2912b99-74bd-4da8-a699-9b11b68c5a0c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:40.924398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e2912b99-74bd-4da8-a699-9b11b68c5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:41.126448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 837/1662] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:41.135600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f13b1-74ec-4846-b096-43e8f070ded2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:41.136423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f13b1-74ec-4846-b096-43e8f070ded2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:41.136803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f13b1-74ec-4846-b096-43e8f070ded2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:41.211283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ce8f13b1-74ec-4846-b096-43e8f070ded2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:41.212802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ce8f13b1-74ec-4846-b096-43e8f070ded2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:41.237424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 826/1663] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:41.244534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2912b99-74bd-4da8-a699-9b11b68c5a0c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:41.277675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2912b99-74bd-4da8-a699-9b11b68c5a0c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:41.295859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:41.296475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:41.296752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:41.297025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:41.298110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:41.298921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:41.313350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:41.328423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:41.342126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f13b1-74ec-4846-b096-43e8f070ded2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:41.347310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 838/1664] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:41.354931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f13b1-74ec-4846-b096-43e8f070ded2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:41.390343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:41.391035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:41.391316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:41.391588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:41.392309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:41.407282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:41.506089 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:41.509649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 827/1665] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:41 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:38:41.542657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a31f25-65b9-4f2c-b6e5-e42be985d7bc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:41.753896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d0305-354c-4639-96fe-549af22370de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:41.755408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d0305-354c-4639-96fe-549af22370de admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:41.755839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d0305-354c-4639-96fe-549af22370de admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:41.756257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d0305-354c-4639-96fe-549af22370de admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:41.757435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d0305-354c-4639-96fe-549af22370de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:41.763309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d0305-354c-4639-96fe-549af22370de admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:41.850306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 839/1666] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:41 2022] POST /identity/v3/users => generated 285 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:41.854643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:41.886430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8060b7-4112-4ad9-a918-f6f89761b424 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:41.899513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:41.902786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:41.903343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:41.904492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:41.905402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:41.912148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:41.953137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 828/1667] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:41.956956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d0305-354c-4639-96fe-549af22370de admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:41.983362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d0305-354c-4639-96fe-549af22370de admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:42.003136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:42.004284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.004284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.004514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.005149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:42.006823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:42.028482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:42.030749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:42.052470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 840/1668] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:42.055380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:42.061881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f31554-6683-4c50-8769-1cc640da85f8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:42.082518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea53a-4841-456b-9f2a-3d515c8e8e48 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:42.083590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:42.084500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.084788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.085067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] PATH_INFO: `/v3/projects/76d57ff0aad740d49fc347486e0c5b22` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.086827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d57ff0aad740d49fc347486e0c5b22)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:42.089453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:42.091588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:42.116130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969765b9-a246-4d85-a192-8ee82433055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:42.116502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969765b9-a246-4d85-a192-8ee82433055f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.116796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969765b9-a246-4d85-a192-8ee82433055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.117074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969765b9-a246-4d85-a192-8ee82433055f admin admin] PATH_INFO: `/v3/projects/08b11e09ea0c4010bb8d093882a6a6ce/users/df781555754b42ce864f8870b212edfc/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.173131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969765b9-a246-4d85-a192-8ee82433055f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b11e09ea0c4010bb8d093882a6a6ce, user_id=df781555754b42ce864f8870b212edfc, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:42.175888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969765b9-a246-4d85-a192-8ee82433055f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:42.212324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 841/1669] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:42 2022] PUT /identity/v3/projects/08b11e09ea0c4010bb8d093882a6a6ce/users/df781555754b42ce864f8870b212edfc/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:42.214238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409e6c2-632b-47d8-9f4b-2a607dfa4d43 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.214541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409e6c2-632b-47d8-9f4b-2a607dfa4d43 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.214820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409e6c2-632b-47d8-9f4b-2a607dfa4d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.233175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d57ff0aad740d49fc347486e0c5b22', 'request_id': 'req-995c4e3f-089e-47be-9fed-019e4c224dba'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:42.264319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d57ff0aad740d49fc347486e0c5b22', 'request_id': 'req-995c4e3f-089e-47be-9fed-019e4c224dba'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:42.283811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d57ff0aad740d49fc347486e0c5b22', 'request_id': 'req-995c4e3f-089e-47be-9fed-019e4c224dba'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:42.286706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2409e6c2-632b-47d8-9f4b-2a607dfa4d43 None None] MFA Rules not processed for user `df781555754b42ce864f8870b212edfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:42.287874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2409e6c2-632b-47d8-9f4b-2a607dfa4d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:42.317971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] The token cache is being invalidate because project 76d57ff0aad740d49fc347486e0c5b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:42.321307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:42.323649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-995c4e3f-089e-47be-9fed-019e4c224dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:42.337064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 829/1670] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:42 2022] DELETE /identity/v3/projects/76d57ff0aad740d49fc347486e0c5b22 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:42.338923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.339184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.339465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.412912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:42.420535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:42.511233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 842/1671] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:42.516052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77a999-53fc-4fd3-9ebf-61737f2cdea4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.516412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77a999-53fc-4fd3-9ebf-61737f2cdea4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.516702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77a999-53fc-4fd3-9ebf-61737f2cdea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.623898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0f77a999-53fc-4fd3-9ebf-61737f2cdea4 None None] MFA Rules not processed for user `eb171002aa064b4e9c2a21ead8a44229`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:42.623898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0f77a999-53fc-4fd3-9ebf-61737f2cdea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:42.641679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 830/1672] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:42.641679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:42.641679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:42.699905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc8c30-88db-48da-a3cb-bead48c3a519 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:42.711366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063cc60-c694-47bf-8566-3e0c3a3f4e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:42.712059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063cc60-c694-47bf-8566-3e0c3a3f4e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.712338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063cc60-c694-47bf-8566-3e0c3a3f4e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.712609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063cc60-c694-47bf-8566-3e0c3a3f4e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.713341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063cc60-c694-47bf-8566-3e0c3a3f4e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:42.730227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063cc60-c694-47bf-8566-3e0c3a3f4e7b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:42.751669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:42.798753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 831/1673] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:42 2022] POST /identity/v3/projects => generated 369 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:42.800471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d695b-18d6-4cc3-af0d-37c453acbda7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:42.808836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d695b-18d6-4cc3-af0d-37c453acbda7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:42.809121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d695b-18d6-4cc3-af0d-37c453acbda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:42.859880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 843/1674] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:42 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:42.860926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77a999-53fc-4fd3-9ebf-61737f2cdea4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:42.884391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77a999-53fc-4fd3-9ebf-61737f2cdea4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:42.902769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e26d695b-18d6-4cc3-af0d-37c453acbda7 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:42.906687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e26d695b-18d6-4cc3-af0d-37c453acbda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:43.074743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.075389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.075669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.075991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.076705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.082308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.137236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 844/1675] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:42 2022] POST /identity/v3/users => generated 291 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:43.138362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.154225 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f93e4c-8927-4fe7-8c6b-c4b230f38562 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.155386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 832/1676] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:43.156551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d695b-18d6-4cc3-af0d-37c453acbda7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.164063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.164836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.165181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.165465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.167432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.169527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.180502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d695b-18d6-4cc3-af0d-37c453acbda7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.188180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 845/1677] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:43.192671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.194379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.195196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.195576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.195945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.196797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:43.197231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.206281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.206688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666ab6e-a91d-4aec-bfbf-490dde9f74fc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.219004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.219896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:43.220368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.220818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.221234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] PATH_INFO: `/v3/projects/bf455a25a9a64cfe959faaed5af91133/users/a91d5bf3b69b443d849dfe9fa53eec02/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.242719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 833/1678] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:43.248487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.258027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf455a25a9a64cfe959faaed5af91133, user_id=a91d5bf3b69b443d849dfe9fa53eec02, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.263874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.271756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99513392-a08b-4d0a-af4a-a44cdda46cb0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.288693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.291756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.292113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.292404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.294605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.295287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 846/1679] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:43 2022] PUT /identity/v3/projects/bf455a25a9a64cfe959faaed5af91133/users/a91d5bf3b69b443d849dfe9fa53eec02/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:43.300508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.301219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.387749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 834/1680] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:43 2022] POST /identity/v3/projects => generated 357 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:43.402340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.500485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc21ba-99ce-4729-8230-805400a7ad3b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.544046 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:43.600253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28617ee0-863d-4f5d-a3ca-5b8f05f5a86c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.602990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.603733 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.604014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.604287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.605026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.610071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.613633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.614381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.614666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.614941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.617349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.619937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.646085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 847/1681] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:43.648392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.676365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 835/1682] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:43 2022] POST /identity/v3/users => generated 279 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:43.676840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8544d8-180c-4356-8b0a-eceaebb90a3a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.678933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.690371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec64d45-54aa-4483-b9a0-7fb5f171e5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.691438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec64d45-54aa-4483-b9a0-7fb5f171e5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.691722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec64d45-54aa-4483-b9a0-7fb5f171e5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.692215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec64d45-54aa-4483-b9a0-7fb5f171e5bf admin admin] PATH_INFO: `/v3/projects/bf455a25a9a64cfe959faaed5af91133/users/a91d5bf3b69b443d849dfe9fa53eec02/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.696553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec64d45-54aa-4483-b9a0-7fb5f171e5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf455a25a9a64cfe959faaed5af91133, user_id=a91d5bf3b69b443d849dfe9fa53eec02, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.702422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72b13d-fcd3-479b-ab54-c1d90e096bd0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.703402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec64d45-54aa-4483-b9a0-7fb5f171e5bf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.711209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.712161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.712440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.712749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.713488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.715147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.740995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 848/1683] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:43 2022] PUT /identity/v3/projects/bf455a25a9a64cfe959faaed5af91133/users/a91d5bf3b69b443d849dfe9fa53eec02/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:43.743101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.743392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.745859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.760407 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 836/1684] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:43.762394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.788859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4017e-825f-450c-ada2-96bef6c5b4d5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.813010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] MFA Rules not processed for user `a91d5bf3b69b443d849dfe9fa53eec02`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:43.814361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:43.911161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.911857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.912138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.912412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.913869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.914912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:43.934226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 837/1685] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:43.937263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:43.951202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10696427-445f-4d43-9b19-45bb39f89b35 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:43.957171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050521f3-43ee-4903-923e-ea75bd2f368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:43.958065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050521f3-43ee-4903-923e-ea75bd2f368f admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:43.958349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050521f3-43ee-4903-923e-ea75bd2f368f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:43.958626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050521f3-43ee-4903-923e-ea75bd2f368f admin admin] PATH_INFO: `/v3/projects/50cb4db522f24a3e8dd7efc51499db96/users/f20ce154d4da4e7e8719b31ae0382b27/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:43.988250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050521f3-43ee-4903-923e-ea75bd2f368f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cb4db522f24a3e8dd7efc51499db96, user_id=f20ce154d4da4e7e8719b31ae0382b27, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:43.990659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 849/1686] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:43 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:43.991466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050521f3-43ee-4903-923e-ea75bd2f368f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:44.001157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:44.003071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:44.014664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 838/1687] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:43 2022] PUT /identity/v3/projects/50cb4db522f24a3e8dd7efc51499db96/users/f20ce154d4da4e7e8719b31ae0382b27/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:44.024807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:44.025104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:44.025421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:44.085041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] MFA Rules not processed for user `f20ce154d4da4e7e8719b31ae0382b27`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:44.086136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:44.113667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:44.121996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60d31-8bf4-49ea-b27b-acb1150c577c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:44.254319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:44.254319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:44.254319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:44.254319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:44.259153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 839/1688] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:44 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:44.260073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:44.262810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:44.270845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:44.372676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:44.376332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:44.409482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 850/1689] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:43 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:38:44.411535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:44.443219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252942c-ac35-4fbc-a692-babd62956b9e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:44.465992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:44.467324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:44.467748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:44.468732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] PATH_INFO: `/v3/users/311061eeaed24854aaa3cfaccc3a4af0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:44.478273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=311061eeaed24854aaa3cfaccc3a4af0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:44.482489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 17 11:38:44.482489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 17 11:38:44.482489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: Jun 17 11:38:44.485387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:44.532836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847d823-a590-44cd-abaa-ad76e11d4815 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:44.556948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:44.558062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:44.560463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:44.560843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:44.561709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:44.566434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:44.659684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 851/1690] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:44 2022] POST /identity/v3/projects => generated 355 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:44.668873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:44.671146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '311061eeaed24854aaa3cfaccc3a4af0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:44.749163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '311061eeaed24854aaa3cfaccc3a4af0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:44.777233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '311061eeaed24854aaa3cfaccc3a4af0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:44.778170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '311061eeaed24854aaa3cfaccc3a4af0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:44.833496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a52cad83-29cb-41dd-a2be-562a01e68bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '311061eeaed24854aaa3cfaccc3a4af0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:44.870727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 840/1691] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:44 2022] DELETE /identity/v3/users/311061eeaed24854aaa3cfaccc3a4af0 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:44.875427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:44.875724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:44.882461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:44.916312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3096b-def9-43c0-aab6-34cb156b4fcf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:44.965348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:44.969667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:44.996720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:45.068359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5ca8d-5497-4991-b2e7-ac9837913d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:45.069185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5ca8d-5497-4991-b2e7-ac9837913d9f admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.069565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5ca8d-5497-4991-b2e7-ac9837913d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.070010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5ca8d-5497-4991-b2e7-ac9837913d9f admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.070898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5ca8d-5497-4991-b2e7-ac9837913d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:45.078763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5ca8d-5497-4991-b2e7-ac9837913d9f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:45.115588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:45.154307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 852/1692] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:44 2022] POST /identity/v3/users => generated 284 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:45.155848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b84c0-b5a5-465c-b837-72d7c2ffb38d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.156134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b84c0-b5a5-465c-b837-72d7c2ffb38d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.156411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b84c0-b5a5-465c-b837-72d7c2ffb38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.225000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-992b84c0-b5a5-465c-b837-72d7c2ffb38d None None] MFA Rules not processed for user `6e2a2c6831ba4d13aeb6e550f8eff9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:45.226958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-992b84c0-b5a5-465c-b837-72d7c2ffb38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:45.254912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 841/1693] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:45.256158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:45.259153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:45.296506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a880d-0e2a-46c7-989a-6f21ac59264c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:45.303674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:45.306296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.307875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.308247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.309763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:45.313875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:45.343120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 842/1694] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:45.346872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:45.350616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:45.450841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 853/1695] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:45.451780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b84c0-b5a5-465c-b837-72d7c2ffb38d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:45.477551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b84c0-b5a5-465c-b837-72d7c2ffb38d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:45.496505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863caaa-9269-4f2a-b402-411fc2578ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:45.497734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863caaa-9269-4f2a-b402-411fc2578ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.498358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863caaa-9269-4f2a-b402-411fc2578ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.499097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863caaa-9269-4f2a-b402-411fc2578ed4 admin admin] PATH_INFO: `/v3/projects/619e67c837494938949f78d238ddd962/users/39f630dcc1b04dd983537f128e5bc7df/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.532120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863caaa-9269-4f2a-b402-411fc2578ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=619e67c837494938949f78d238ddd962, user_id=39f630dcc1b04dd983537f128e5bc7df, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:45.537702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863caaa-9269-4f2a-b402-411fc2578ed4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:45.544309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d759-a69d-44aa-99f8-fc4fe326f2db admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:45.559472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:45.560185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.560474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.560879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.567434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 854/1696] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:45 2022] PUT /identity/v3/projects/619e67c837494938949f78d238ddd962/users/39f630dcc1b04dd983537f128e5bc7df/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:45.570819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.571749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.572660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.595199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:45.596659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:45.617211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] MFA Rules not processed for user `6e2a2c6831ba4d13aeb6e550f8eff9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:45.618714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:45.647799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 843/1697] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:45.653649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:45.692819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c42fe4-0c76-4546-9c23-a74c1f13b0ee service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:45.807178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 855/1698] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:45.829889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:45.834643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:45.868781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:45.873289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.883580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.884054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.885375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:45.888255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:45.901134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 844/1699] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:45.906782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:45.920268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7dd3c-0696-46a7-a463-7bfa2e27b144 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:45.927374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d3aab-f6a4-4a3e-ab7a-e02214bfb7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:45.929021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3aab-f6a4-4a3e-ab7a-e02214bfb7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.929403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3aab-f6a4-4a3e-ab7a-e02214bfb7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.929885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3aab-f6a4-4a3e-ab7a-e02214bfb7d7 admin admin] PATH_INFO: `/v3/projects/619e67c837494938949f78d238ddd962/users/39f630dcc1b04dd983537f128e5bc7df/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:45.933229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d3aab-f6a4-4a3e-ab7a-e02214bfb7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=619e67c837494938949f78d238ddd962, user_id=39f630dcc1b04dd983537f128e5bc7df, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:45.936138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d3aab-f6a4-4a3e-ab7a-e02214bfb7d7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:45.966888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 845/1700] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:45 2022] PUT /identity/v3/projects/619e67c837494938949f78d238ddd962/users/39f630dcc1b04dd983537f128e5bc7df/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:38:45.973105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:45.973426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:45.973709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.004150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:46.059167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] MFA Rules not processed for user `39f630dcc1b04dd983537f128e5bc7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:46.060489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:46.115972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d963f-a0ec-4886-aa9b-8acc1640a362 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:46.206880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:46.277828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:46.278606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.281348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.281348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.288090 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 846/1701] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:45 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:38:46.302843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:46.306667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:46.354690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:46.356743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:46.367340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 856/1702] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:46.368213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:46.370232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:46.468994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0a412-2c10-489e-9801-c7b8c57fccec None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:46.480659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:46.480659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.480659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.480659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.496641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:46.498323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:46.503782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3781706-0789-4e9f-b649-0962d2570b41 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:46.516446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 847/1703] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:46 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:46.530323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:46.531225 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.531517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.532403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] PATH_INFO: `/v3/projects/a957d18da3074f93a64089c5926549e0` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.533776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a957d18da3074f93a64089c5926549e0)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:46.539289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:46.566518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:46.662416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a957d18da3074f93a64089c5926549e0', 'request_id': 'req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:46.692571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a957d18da3074f93a64089c5926549e0', 'request_id': 'req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:46.718519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a957d18da3074f93a64089c5926549e0', 'request_id': 'req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:46.734922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:46.748627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6711702-4691-43ff-b686-b9eccd0f3378 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:46.753867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] The token cache is being invalidate because project a957d18da3074f93a64089c5926549e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:46.754753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:46.756823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3b5cbfe4-8d06-4f9e-b61c-bd3e191460f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:46.762339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 857/1704] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:46 2022] DELETE /identity/v3/projects/a957d18da3074f93a64089c5926549e0 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:46.767244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd06dc6-6a97-4e07-8f3b-0618c1d85c4d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.767661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd06dc6-6a97-4e07-8f3b-0618c1d85c4d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.768061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd06dc6-6a97-4e07-8f3b-0618c1d85c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.817975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4dd06dc6-6a97-4e07-8f3b-0618c1d85c4d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:46.819497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4dd06dc6-6a97-4e07-8f3b-0618c1d85c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:46.929123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:46.929801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.930980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.931943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:46.935797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:46.976380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 858/1705] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:46.986290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28452d-09a9-4385-bdeb-66f4133c98fc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:46.986794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28452d-09a9-4385-bdeb-66f4133c98fc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:46.987318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28452d-09a9-4385-bdeb-66f4133c98fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.016756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.018714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.024686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8f28452d-09a9-4385-bdeb-66f4133c98fc None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:47.027164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8f28452d-09a9-4385-bdeb-66f4133c98fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:47.039328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 848/1706] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:46 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:47.058343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 859/1707] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:47.066549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.098226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc208065-ab73-451b-aeb7-c6cdaa60b17d service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.111590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.112620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.113095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.113579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.114861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:47.117763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.124885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.158704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 849/1708] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:47.162224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28452d-09a9-4385-bdeb-66f4133c98fc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.182463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28452d-09a9-4385-bdeb-66f4133c98fc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.203107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b95ea2-5982-4665-8152-29c42674fa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.203826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b95ea2-5982-4665-8152-29c42674fa4a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.204118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b95ea2-5982-4665-8152-29c42674fa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.204404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b95ea2-5982-4665-8152-29c42674fa4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.205133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b95ea2-5982-4665-8152-29c42674fa4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.207144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b95ea2-5982-4665-8152-29c42674fa4a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.300164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 860/1709] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:47 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:47.303934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.309180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de2c0-2bc2-4afc-967e-84471009ddf6 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.309483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de2c0-2bc2-4afc-967e-84471009ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.309769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52de2c0-2bc2-4afc-967e-84471009ddf6 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.311748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 861/1710] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:38:47.330411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bf216-b6ea-4845-ab31-9eeea65c9e61 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.338056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb6ec0-91a0-4462-acc8-ba09d0a53c9b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.338592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb6ec0-91a0-4462-acc8-ba09d0a53c9b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.339029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb6ec0-91a0-4462-acc8-ba09d0a53c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.377709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9bcb6ec0-91a0-4462-acc8-ba09d0a53c9b None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:47.379370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9bcb6ec0-91a0-4462-acc8-ba09d0a53c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:47.438097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:47.518921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 862/1711] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:38:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:38:47.582652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.583583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.584056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.584506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.585409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.588296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.615622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bcaf1-f8b5-473e-a714-70da9302fe20 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.615915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bcaf1-f8b5-473e-a714-70da9302fe20 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.616198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bcaf1-f8b5-473e-a714-70da9302fe20 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.617921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 863/1712] 10.222.0.25 () {54 vars in 912 bytes} [Fri Jun 17 11:38:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:38:47.626460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.626750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.627035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.663260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 850/1713] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:47 2022] POST /identity/v3/users => generated 284 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:47.667024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.673237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:47.686950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1aa6b-5433-4a77-abdf-d3c8b314fa5a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.705393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.706370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.706839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.707361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.708257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.710500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.725427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] MFA Rules not processed for user `b3b35d6d2cd848f4841352fbb58f2053`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:47.727413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:47.737068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 851/1714] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:38:47.742030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.766381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51526ae4-4f70-43af-91ca-90d2315c3bb4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.782842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.784540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.784984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.785404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.786549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.788395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.819269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 852/1715] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:47.824105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.843047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54387e95-964d-4607-8d9e-397ff81d68c1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.850363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 864/1716] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:38:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:47.851339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.852684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:47.854932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.858393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.858916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.859389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] PATH_INFO: `/v3/projects/e9ff424428684c538a9402e9bfca41cc/users/2e33b36670fd46528f6225894c6ff322/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.871595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ec49-a07c-41a6-82bf-fa750b5a45f4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:47.886140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:47.886926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:47.887273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:47.887562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] PATH_INFO: `/v3/users/df781555754b42ce864f8870b212edfc` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:47.889436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df781555754b42ce864f8870b212edfc)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.890318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ff424428684c538a9402e9bfca41cc, user_id=2e33b36670fd46528f6225894c6ff322, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:47.893189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.894953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:47.918583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 853/1717] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:47 2022] PUT /identity/v3/projects/e9ff424428684c538a9402e9bfca41cc/users/2e33b36670fd46528f6225894c6ff322/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:47.919933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:47.921971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:47.997456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df781555754b42ce864f8870b212edfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:48.019765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df781555754b42ce864f8870b212edfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:48.031157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df781555754b42ce864f8870b212edfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:48.031925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df781555754b42ce864f8870b212edfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:48.053779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ca8de13e-fd25-4946-bcbe-c7e9e35251f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df781555754b42ce864f8870b212edfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:48.063623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 865/1718] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:47 2022] DELETE /identity/v3/users/df781555754b42ce864f8870b212edfc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:48.064879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97226ed8-422e-4ed1-8b12-5b354c35604e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.065081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97226ed8-422e-4ed1-8b12-5b354c35604e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.065366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97226ed8-422e-4ed1-8b12-5b354c35604e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.066990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 866/1719] 10.222.0.25 () {54 vars in 960 bytes} [Fri Jun 17 11:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:48.068053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ea293-6149-4cf1-974b-4690a0d6af3a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.069101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ea293-6149-4cf1-974b-4690a0d6af3a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.069555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ea293-6149-4cf1-974b-4690a0d6af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.082030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b10a8-f9c2-43fd-809e-abdb8cc996d6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:48.101497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e34ea293-6149-4cf1-974b-4690a0d6af3a None None] MFA Rules not processed for user `2e33b36670fd46528f6225894c6ff322`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:48.102898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e34ea293-6149-4cf1-974b-4690a0d6af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.181735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f417fa7-6d25-4d47-ad08-93b595bb279f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:48.182314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f417fa7-6d25-4d47-ad08-93b595bb279f service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.182601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f417fa7-6d25-4d47-ad08-93b595bb279f service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.182880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f417fa7-6d25-4d47-ad08-93b595bb279f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.185358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f417fa7-6d25-4d47-ad08-93b595bb279f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.188094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 867/1720] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:48.192485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.192742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.192986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.215063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f417fa7-6d25-4d47-ad08-93b595bb279f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:48.216495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:48.216602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f417fa7-6d25-4d47-ad08-93b595bb279f service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:48.224395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:48.226565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.227219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 854/1721] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:48.231675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963fbc-3736-420a-a2c3-bb8dd7b9974e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.232094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963fbc-3736-420a-a2c3-bb8dd7b9974e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.232380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963fbc-3736-420a-a2c3-bb8dd7b9974e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.274853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d6963fbc-3736-420a-a2c3-bb8dd7b9974e None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:48.292430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d6963fbc-3736-420a-a2c3-bb8dd7b9974e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.317749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:48.389802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 868/1722] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:48.393391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:48.395704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.423812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 855/1723] 10.222.0.25 () {58 vars in 1057 bytes} [Fri Jun 17 11:38:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:48.434816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963fbc-3736-420a-a2c3-bb8dd7b9974e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:48.456778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963fbc-3736-420a-a2c3-bb8dd7b9974e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:48.462218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:48.463420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.463704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.463980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.476030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.500113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:48.502027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:48.521562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 856/1724] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:48.530906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:48.534889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:48.543181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97394949-0724-43ab-8655-efbc0945e8e4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:48.552722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa825643-4551-4251-a43a-12b5c8b78651 service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:48.553330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b7a6a-d7e2-4e40-892d-ae5ca7453a13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:48.554266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7a6a-d7e2-4e40-892d-ae5ca7453a13 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.554342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7a6a-d7e2-4e40-892d-ae5ca7453a13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.554595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7a6a-d7e2-4e40-892d-ae5ca7453a13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.557169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:48.557794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:48.558113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:48.558392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:48.569582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:48.571017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:48.576063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b7a6a-d7e2-4e40-892d-ae5ca7453a13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:48.585537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 857/1725] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:48.591979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b7a6a-d7e2-4e40-892d-ae5ca7453a13 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:48.617343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 869/1726] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:49.547606 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:49.571381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e812b-edfa-4a72-acb7-c9c10346d30b service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:49.606560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85e14c-870a-4bda-aca3-34729b3e51b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:49.608047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e14c-870a-4bda-aca3-34729b3e51b3 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:49.608536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e14c-870a-4bda-aca3-34729b3e51b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:49.608986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e14c-870a-4bda-aca3-34729b3e51b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:49.616648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8f85e14c-870a-4bda-aca3-34729b3e51b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:49.653517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85e14c-870a-4bda-aca3-34729b3e51b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:49.666419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85e14c-870a-4bda-aca3-34729b3e51b3 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:49.693491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 858/1727] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:51.192645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0c542-4a45-4558-a235-3f5f6de77136 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:51.193102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0c542-4a45-4558-a235-3f5f6de77136 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:51.193228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0c542-4a45-4558-a235-3f5f6de77136 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:51.194810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 870/1728] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:38:51.221137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:51.221650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:51.221750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:51.269326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] MFA Rules not processed for user `1c8bdac27bde4a2785d4aaad8213baca`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:51.277586 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:51.313534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 859/1729] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:38:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:51.395293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0c542-4a45-4558-a235-3f5f6de77136 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:51.399925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5db0c542-4a45-4558-a235-3f5f6de77136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:51.447737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0c542-4a45-4558-a235-3f5f6de77136 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:51.479028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:51.479850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:51.480153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:51.480430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:51.500302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:51.503912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:51.528351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 871/1730] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:51.578552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:51.580441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:51.605134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7c7a6-8354-40d3-8305-cd6939fdefcb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:51.645259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:51.647679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:51.648083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:51.652729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] PATH_INFO: `/v3/users/4a66f80e4c814767bca999f00e95511e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:51.663636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a66f80e4c814767bca999f00e95511e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:51.693554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:51.848757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a66f80e4c814767bca999f00e95511e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:51.875637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a66f80e4c814767bca999f00e95511e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:51.903581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a66f80e4c814767bca999f00e95511e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:51.904014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a66f80e4c814767bca999f00e95511e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:51.919456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a66f80e4c814767bca999f00e95511e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:51.930108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 860/1731] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:51 2022] DELETE /identity/v3/users/4a66f80e4c814767bca999f00e95511e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:51.953762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:51.955513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:51.980860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40612814-ca83-4a4a-b436-ee193c46e843 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:52.110931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:52.111875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:52.112158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:52.112429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] PATH_INFO: `/v3/projects/08b11e09ea0c4010bb8d093882a6a6ce` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:52.114536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b11e09ea0c4010bb8d093882a6a6ce)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:52.118139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:52.215268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:52.224330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b11e09ea0c4010bb8d093882a6a6ce', 'request_id': 'req-f1463819-1d07-42ae-826e-df1bef59accb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.251535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b11e09ea0c4010bb8d093882a6a6ce', 'request_id': 'req-f1463819-1d07-42ae-826e-df1bef59accb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.278308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b11e09ea0c4010bb8d093882a6a6ce', 'request_id': 'req-f1463819-1d07-42ae-826e-df1bef59accb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.314449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] The token cache is being invalidate because project 08b11e09ea0c4010bb8d093882a6a6ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:52.314854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.316184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.320238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 872/1732] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:51 2022] DELETE /identity/v3/projects/08b11e09ea0c4010bb8d093882a6a6ce => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:52.327187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:52.329316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:52.384128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9e1ee-aa31-4579-9f11-ed84431fa6e2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:52.423421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:52.430581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:52.579940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:52.652712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:52.653606 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:52.654233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:52.654802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] PATH_INFO: `/v3/users/a91d5bf3b69b443d849dfe9fa53eec02` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:52.656668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91d5bf3b69b443d849dfe9fa53eec02)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:52.659386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:52.682170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1463819-1d07-42ae-826e-df1bef59accb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:52.697980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:52.698906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:52.699165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:52.702119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] PATH_INFO: `/v3/projects/4bf1279fc40c41dd8f652e1d80bb0314` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:52.704293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf1279fc40c41dd8f652e1d80bb0314)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:52.707215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:52.916813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf1279fc40c41dd8f652e1d80bb0314', 'request_id': 'req-daa855bc-aa55-45a2-a74e-5101ed0b77fa'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.928899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91d5bf3b69b443d849dfe9fa53eec02'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.953691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf1279fc40c41dd8f652e1d80bb0314', 'request_id': 'req-daa855bc-aa55-45a2-a74e-5101ed0b77fa'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.970701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91d5bf3b69b443d849dfe9fa53eec02'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:52.980535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf1279fc40c41dd8f652e1d80bb0314', 'request_id': 'req-daa855bc-aa55-45a2-a74e-5101ed0b77fa'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.010016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] The token cache is being invalidate because project 4bf1279fc40c41dd8f652e1d80bb0314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:53.010629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.012749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.016336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91d5bf3b69b443d849dfe9fa53eec02'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.017202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91d5bf3b69b443d849dfe9fa53eec02'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.017646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 873/1733] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:52 2022] DELETE /identity/v3/projects/4bf1279fc40c41dd8f652e1d80bb0314 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:53.026091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:53.028864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:53.057232 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9197f67a-775f-4e5a-b78f-3a5f9f1ccc1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91d5bf3b69b443d849dfe9fa53eec02'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.076963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 861/1734] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:52 2022] DELETE /identity/v3/users/a91d5bf3b69b443d849dfe9fa53eec02 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:53.207462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:53.216391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa855bc-aa55-45a2-a74e-5101ed0b77fa admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:53.446716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:53.447826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:53.448117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:53.448409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] PATH_INFO: `/v3/users/eb171002aa064b4e9c2a21ead8a44229` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:53.451152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb171002aa064b4e9c2a21ead8a44229)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:53.457578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:53.594439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb171002aa064b4e9c2a21ead8a44229'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.614431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:53.614956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:53.615138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:53.621420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb171002aa064b4e9c2a21ead8a44229'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.630509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb171002aa064b4e9c2a21ead8a44229'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.631507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb171002aa064b4e9c2a21ead8a44229'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.651828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb171002aa064b4e9c2a21ead8a44229'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:53.666864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 874/1735] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:38:53 2022] DELETE /identity/v3/users/eb171002aa064b4e9c2a21ead8a44229 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:38:53.706508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:53.707958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:53.870794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 862/1736] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:38:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:54.113924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:54.117661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:54.306600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057db9e-9718-4eb1-86d2-5cef9e0e8bc5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:54.340293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:54.341930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:54.342293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:54.343320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:54.362180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:54.365821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:54.411094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 875/1737] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:38:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:38:55.604577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:55.606352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:55.859209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85549e0f-83a9-40a1-99dc-50e7ac258e4a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:55.886028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:55.887009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:55.889429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:55.890290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] PATH_INFO: `/v3/projects/8b7b79252e75415e97e56b49392be147` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:55.894901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7b79252e75415e97e56b49392be147)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:55.896879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:55.919730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:55.924695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:55.983722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ea62c-e6a2-473b-9d09-531785f03a73 service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:55.992626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:56.004050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:56.005624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.008338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.008841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] PATH_INFO: `/v3/projects/bf455a25a9a64cfe959faaed5af91133` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.011851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf455a25a9a64cfe959faaed5af91133)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:56.016433 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:56.043376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:56.174013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7b79252e75415e97e56b49392be147', 'request_id': 'req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.189052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf455a25a9a64cfe959faaed5af91133', 'request_id': 'req-74a76942-1c60-45ca-8b54-abc613c069d5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.204250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7b79252e75415e97e56b49392be147', 'request_id': 'req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.210204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf455a25a9a64cfe959faaed5af91133', 'request_id': 'req-74a76942-1c60-45ca-8b54-abc613c069d5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.215062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7b79252e75415e97e56b49392be147', 'request_id': 'req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.218200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf455a25a9a64cfe959faaed5af91133', 'request_id': 'req-74a76942-1c60-45ca-8b54-abc613c069d5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.242909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] The token cache is being invalidate because project bf455a25a9a64cfe959faaed5af91133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:56.243658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.245286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74a76942-1c60-45ca-8b54-abc613c069d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.256652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] The token cache is being invalidate because project 8b7b79252e75415e97e56b49392be147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:38:56.257130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.260035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-52add7e5-5e3c-4a48-9fd5-8b4283c637b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:38:56.263448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 863/1738] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:55 2022] DELETE /identity/v3/projects/8b7b79252e75415e97e56b49392be147 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:56.264926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167853-a2df-44f2-afa3-0583444bc7c5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.266427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167853-a2df-44f2-afa3-0583444bc7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.267247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc167853-a2df-44f2-afa3-0583444bc7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.271968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 876/1739] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:38:55 2022] DELETE /identity/v3/projects/bf455a25a9a64cfe959faaed5af91133 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:38:56.276869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb618c-5082-4254-951b-23223356974a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.277219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb618c-5082-4254-951b-23223356974a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.277515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb618c-5082-4254-951b-23223356974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.339697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bc167853-a2df-44f2-afa3-0583444bc7c5 None None] MFA Rules not processed for user `b3b35d6d2cd848f4841352fbb58f2053`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:56.347098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bc167853-a2df-44f2-afa3-0583444bc7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:56.369241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4afb618c-5082-4254-951b-23223356974a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:56.373615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4afb618c-5082-4254-951b-23223356974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:56.495459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 864/1740] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Jun 17 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:56.497177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0631a3-ecbc-4956-bf65-45db34dbf039 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.497943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0631a3-ecbc-4956-bf65-45db34dbf039 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.500598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0631a3-ecbc-4956-bf65-45db34dbf039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.548575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ce0631a3-ecbc-4956-bf65-45db34dbf039 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:56.551355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ce0631a3-ecbc-4956-bf65-45db34dbf039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:56.610494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 877/1741] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:56.611660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb618c-5082-4254-951b-23223356974a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:56.617731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4afb618c-5082-4254-951b-23223356974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:56.660580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb618c-5082-4254-951b-23223356974a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:56.671533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba1b0e-ce99-471d-9964-921bcaacbdd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:56.674809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba1b0e-ce99-471d-9964-921bcaacbdd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.680080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba1b0e-ce99-471d-9964-921bcaacbdd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.680080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba1b0e-ce99-471d-9964-921bcaacbdd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.706631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba1b0e-ce99-471d-9964-921bcaacbdd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:56.722441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba1b0e-ce99-471d-9964-921bcaacbdd0 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:56.741358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 865/1742] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:56.744514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37b5ea-4f3a-44b5-87d8-33141375c11a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.745934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37b5ea-4f3a-44b5-87d8-33141375c11a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.746377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37b5ea-4f3a-44b5-87d8-33141375c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.758454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 878/1743] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:56.761393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22558-64b5-48c3-a0d7-ed9c4f95feb3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.765236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22558-64b5-48c3-a0d7-ed9c4f95feb3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.771374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22558-64b5-48c3-a0d7-ed9c4f95feb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.785584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9e37b5ea-4f3a-44b5-87d8-33141375c11a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:56.789269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9e37b5ea-4f3a-44b5-87d8-33141375c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:56.802685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 866/1744] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:56.806769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37b5ea-4f3a-44b5-87d8-33141375c11a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:56.822710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37b5ea-4f3a-44b5-87d8-33141375c11a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:56.830221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:56.831152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.831575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.831990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.833178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d9d22558-64b5-48c3-a0d7-ed9c4f95feb3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:56.834488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:56.836607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:56.840434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:56.841321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d9d22558-64b5-48c3-a0d7-ed9c4f95feb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:56.873545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 879/1745] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:56.874494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22558-64b5-48c3-a0d7-ed9c4f95feb3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:56.876779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 867/1746] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:56.903643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22558-64b5-48c3-a0d7-ed9c4f95feb3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:56.932992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:56.946062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:56.948174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.949164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.953374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.954372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:38:56.959677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:56.960700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:56.970895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bacf4-f30d-4d12-8fa7-b2a96fe6cd8f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:56.986548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:56.988771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:56.989231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:56.989629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:56.991967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:56.995999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.020464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 880/1747] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:38:57.024744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.060716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 868/1748] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:56 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:57.064491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.073127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103676d-0cb5-4260-94d1-eb3b540c32ca admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.075603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:57.167044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2716fd1-ba4a-4c99-96a4-38a965492469 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.176639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.177515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.177938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.178220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.179390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.182752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.231285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 869/1749] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:57 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:38:57.235615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.236355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.237274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.237568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.237880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.238600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.240658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.267534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:57.268154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4db7-dfb0-489d-aeb4-8e38b003b240 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.287295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbfe8d4-057a-4681-83be-e93b8bd3b7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.289584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfe8d4-057a-4681-83be-e93b8bd3b7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.290063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfe8d4-057a-4681-83be-e93b8bd3b7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.295184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfe8d4-057a-4681-83be-e93b8bd3b7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.296223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfe8d4-057a-4681-83be-e93b8bd3b7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.298074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbfe8d4-057a-4681-83be-e93b8bd3b7d3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.303707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 881/1750] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:57 2022] POST /identity/v3/projects => generated 407 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:57.304769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.334416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 870/1751] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:57.338247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa2cd-6768-477c-8532-5da5a91c6a00 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.338804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa2cd-6768-477c-8532-5da5a91c6a00 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.339226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316aa2cd-6768-477c-8532-5da5a91c6a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.386616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-316aa2cd-6768-477c-8532-5da5a91c6a00 None None] MFA Rules not processed for user `3a0f25c7ce194ddbb21d1c8b97d03c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:57.388397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-316aa2cd-6768-477c-8532-5da5a91c6a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:57.429507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5c570-5adf-4849-8d75-62bbfa2cd5eb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.437927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.439114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.439852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.440420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.441472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.443923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.487793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 882/1752] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:57 2022] POST /identity/v3/users => generated 304 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:57.492864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.522823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48be98b-d1ca-45d5-a1a5-e49b122f6ab3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.536675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.536892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.537215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.537215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.537215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.537215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.544126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 871/1753] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:57.545373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa2cd-6768-477c-8532-5da5a91c6a00 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.558801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316aa2cd-6768-477c-8532-5da5a91c6a00 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.566131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.566872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 883/1754] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:57.567402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.567935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.568220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.568645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.569604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.570188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:57.571768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.583448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 872/1755] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:57.585142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.612050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90950f-8766-43cc-9dba-09ca0bb72522 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.626693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.632148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.632148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.635044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] PATH_INFO: `/v3/projects/0af8188ddb76467da2ed1183f127d5f8/users/8f3f7c77b5204d43afac5996a1d587f8/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.675511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af8188ddb76467da2ed1183f127d5f8, user_id=8f3f7c77b5204d43afac5996a1d587f8, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.678944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:57.711138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 873/1756] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:57 2022] PUT /identity/v3/projects/0af8188ddb76467da2ed1183f127d5f8/users/8f3f7c77b5204d43afac5996a1d587f8/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:57.712670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:57.751737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21520a-63c3-43ff-9023-0463083806a9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.852442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:57.862140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:57.878581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.879392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.879535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.879810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.933162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12489d-2aa2-4137-9358-64982e91160a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:57.945725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f79a06-a3d5-495f-9f15-3950ef860002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:57.946884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f79a06-a3d5-495f-9f15-3950ef860002 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:57.947285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f79a06-a3d5-495f-9f15-3950ef860002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:57.947667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f79a06-a3d5-495f-9f15-3950ef860002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:57.949984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f79a06-a3d5-495f-9f15-3950ef860002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:57.954213 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f79a06-a3d5-495f-9f15-3950ef860002 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:58.009469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 874/1757] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:38:58.013040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c5926-4d3e-4d44-b1f7-06eb52c40405 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:58.013195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c5926-4d3e-4d44-b1f7-06eb52c40405 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:58.013591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c5926-4d3e-4d44-b1f7-06eb52c40405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:58.094429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:58.100821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:58.117393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 884/1758] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:58.119367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:58.149139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5501917-6514-40c7-b765-57d82b7f2a2e service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:58.161090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6661bc-b95e-4e16-ad02-3e701da5380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:58.167445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6661bc-b95e-4e16-ad02-3e701da5380f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:58.167776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6661bc-b95e-4e16-ad02-3e701da5380f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:58.168058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6661bc-b95e-4e16-ad02-3e701da5380f admin admin] PATH_INFO: `/v3/projects/3f4ecfc760014c57a09e0e5a5c769e04/users/23d3e173da5b4430a47adcc76d025261/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:58.181079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8f0c5926-4d3e-4d44-b1f7-06eb52c40405 None None] MFA Rules not processed for user `8f3f7c77b5204d43afac5996a1d587f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:58.192984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8f0c5926-4d3e-4d44-b1f7-06eb52c40405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:58.221700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6661bc-b95e-4e16-ad02-3e701da5380f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f4ecfc760014c57a09e0e5a5c769e04, user_id=23d3e173da5b4430a47adcc76d025261, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:58.241241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6661bc-b95e-4e16-ad02-3e701da5380f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:58.289784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 885/1759] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:38:58 2022] PUT /identity/v3/projects/3f4ecfc760014c57a09e0e5a5c769e04/users/23d3e173da5b4430a47adcc76d025261/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:38:58.324152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fef0d-4c54-4c34-869a-e3abda022cde None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:58.325023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fef0d-4c54-4c34-869a-e3abda022cde None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:58.325482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fef0d-4c54-4c34-869a-e3abda022cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:58.395339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b26fef0d-4c54-4c34-869a-e3abda022cde None None] MFA Rules not processed for user `23d3e173da5b4430a47adcc76d025261`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:58.398635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b26fef0d-4c54-4c34-869a-e3abda022cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:58.520720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 875/1760] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:58.529431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5b5a7-f3c4-400e-b123-320a890e665e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:58.529871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5b5a7-f3c4-400e-b123-320a890e665e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:58.530754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5b5a7-f3c4-400e-b123-320a890e665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:58.703264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bec5b5a7-f3c4-400e-b123-320a890e665e None None] MFA Rules not processed for user `c2c930cfab47464fb784c98f58abbb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:38:58.704537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bec5b5a7-f3c4-400e-b123-320a890e665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:58.843730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 886/1761] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:58.843730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fef0d-4c54-4c34-869a-e3abda022cde None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:58.989832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 876/1762] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:38:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:38:58.993093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5b5a7-f3c4-400e-b123-320a890e665e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:59.137815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fef0d-4c54-4c34-869a-e3abda022cde None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:59.161173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:59.161886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:59.162173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:59.162449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:59.164510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:59.176456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:59.250994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:59.286980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 887/1763] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:58 2022] POST /identity/v3/projects => generated 387 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:38:59.287806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:59.291960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:38:59.340727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:38:59.415012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5b5a7-f3c4-400e-b123-320a890e665e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:59.509388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d148-f0dc-4062-a1e0-797d4da44710 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:59.526563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:59.527323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:59.527695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:59.528118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:59.823406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc0218-ac2f-4d6c-bc45-d768f369e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:59.828658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0218-ac2f-4d6c-bc45-d768f369e9bc admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:59.829089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0218-ac2f-4d6c-bc45-d768f369e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:59.830169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc0218-ac2f-4d6c-bc45-d768f369e9bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:59.833645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc0218-ac2f-4d6c-bc45-d768f369e9bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:59.841180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc0218-ac2f-4d6c-bc45-d768f369e9bc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:59.851534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:38:59.853695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:38:59.909673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 888/1764] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:38:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:38:59.910512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:38:59.923011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051de68-3648-4daa-a24c-7d80ab27d298 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:38:59.961630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15129b6-a870-4aa2-bcb3-ab26d3e29f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:38:59.970157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15129b6-a870-4aa2-bcb3-ab26d3e29f3d admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:38:59.970538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15129b6-a870-4aa2-bcb3-ab26d3e29f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:38:59.970839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15129b6-a870-4aa2-bcb3-ab26d3e29f3d admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:38:59.971566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15129b6-a870-4aa2-bcb3-ab26d3e29f3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.001975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15129b6-a870-4aa2-bcb3-ab26d3e29f3d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.003829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 877/1765] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:38:58 2022] POST /identity/v3/projects => generated 409 bytes in 1013 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:00.015004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc22df-794d-41c0-8b15-977752d520d9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.020036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc22df-794d-41c0-8b15-977752d520d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.022346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc22df-794d-41c0-8b15-977752d520d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.107408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 889/1766] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:38:59 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:00.108994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce26f4-d331-4837-b421-9b6c49e10d83 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.109333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce26f4-d331-4837-b421-9b6c49e10d83 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.109608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce26f4-d331-4837-b421-9b6c49e10d83 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.111073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 890/1767] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:00.111878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce26f4-d331-4837-b421-9b6c49e10d83 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:00.210401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2fdc22df-794d-41c0-8b15-977752d520d9 None None] MFA Rules not processed for user `9e209f3667f54c2b9bc6eab13010296c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:00.219504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2fdc22df-794d-41c0-8b15-977752d520d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:00.256265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce26f4-d331-4837-b421-9b6c49e10d83 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:00.286706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:00.287696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.288108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.288513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.289462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.292717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.343880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 891/1768] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:00 2022] POST /identity/v3/users => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:00.345973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:00.362565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:00.373321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93d741-cc21-4a3f-beeb-e8e213c9cf18 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:00.387473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:00.389614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.390466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.390753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.391695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.397266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.425239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 892/1769] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:00.426579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:00.444999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603014cd-b592-437d-ab32-fbec7475d023 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:00.454730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139d42c-ef25-4edc-a641-b8e3dd6d4ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:00.455756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139d42c-ef25-4edc-a641-b8e3dd6d4ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.455977 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139d42c-ef25-4edc-a641-b8e3dd6d4ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.456277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139d42c-ef25-4edc-a641-b8e3dd6d4ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.457140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139d42c-ef25-4edc-a641-b8e3dd6d4ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.460902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139d42c-ef25-4edc-a641-b8e3dd6d4ce0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.484249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 893/1770] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:00.486973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e50180-413d-48a9-a576-72a73e011e75 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.487285 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e50180-413d-48a9-a576-72a73e011e75 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.487618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e50180-413d-48a9-a576-72a73e011e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.510581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 878/1771] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:00.522836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc22df-794d-41c0-8b15-977752d520d9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:00.525570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-54e50180-413d-48a9-a576-72a73e011e75 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:00.527168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-54e50180-413d-48a9-a576-72a73e011e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:00.551410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc22df-794d-41c0-8b15-977752d520d9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:00.564945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:00.565832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.566255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.566634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.568168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.570025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.591682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 879/1772] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:00.592751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:00.628025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18b2c9-38bd-4d38-a4b4-672e81a537d7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:00.637477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a28a55-2155-4b4c-927a-60f517c3ea63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:00.638515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a28a55-2155-4b4c-927a-60f517c3ea63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.639011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a28a55-2155-4b4c-927a-60f517c3ea63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.639481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a28a55-2155-4b4c-927a-60f517c3ea63 admin admin] PATH_INFO: `/v3/projects/eda43add11df48628365612b41ceaacc/users/3a7fb38eef9f4b0ab3b7ab56718fe20c/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.674897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a28a55-2155-4b4c-927a-60f517c3ea63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda43add11df48628365612b41ceaacc, user_id=3a7fb38eef9f4b0ab3b7ab56718fe20c, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.679013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a28a55-2155-4b4c-927a-60f517c3ea63 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.707139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 880/1773] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:00 2022] PUT /identity/v3/projects/eda43add11df48628365612b41ceaacc/users/3a7fb38eef9f4b0ab3b7ab56718fe20c/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:00.709563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054661e-b89a-4fa3-af69-bbe07abfbd81 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.711245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054661e-b89a-4fa3-af69-bbe07abfbd81 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.711791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054661e-b89a-4fa3-af69-bbe07abfbd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.727618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 894/1774] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:39:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:00.729029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e50180-413d-48a9-a576-72a73e011e75 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:00.744251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e50180-413d-48a9-a576-72a73e011e75 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:00.748541 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1054661e-b89a-4fa3-af69-bbe07abfbd81 None None] MFA Rules not processed for user `9e209f3667f54c2b9bc6eab13010296c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:00.752733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1054661e-b89a-4fa3-af69-bbe07abfbd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:00.829985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:00.896747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:00.899235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.901577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.902061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] PATH_INFO: `/v3/projects/2943282a396c472b8191113b7960106a/users/ef3177a8ead542f1800433cf7b377212/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.918386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 881/1775] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:00.925789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054661e-b89a-4fa3-af69-bbe07abfbd81 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:00.944021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2943282a396c472b8191113b7960106a, user_id=ef3177a8ead542f1800433cf7b377212, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.950352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:00.950913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054661e-b89a-4fa3-af69-bbe07abfbd81 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:00.966885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:00.967600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:00.967947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:00.968319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:00.969202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:00.970972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:01.019830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 895/1776] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:00 2022] PUT /identity/v3/projects/2943282a396c472b8191113b7960106a/users/ef3177a8ead542f1800433cf7b377212/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:01.020699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:01.045006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 882/1777] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:01.045989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:01.352290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:01.436887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ebc49-0379-4b1b-be39-dca9109d12e4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:01.440679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f06ae-db6c-431e-9b3a-6ca2f4005252 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:01.458737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:01.461391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:01.463080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577d979-5ad1-4c85-9380-87c5531540dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:01.465163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:01.466193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577d979-5ad1-4c85-9380-87c5531540dc service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:01.466725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:01.467730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577d979-5ad1-4c85-9380-87c5531540dc service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:01.468080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577d979-5ad1-4c85-9380-87c5531540dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:01.504687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577d979-5ad1-4c85-9380-87c5531540dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:01.508925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577d979-5ad1-4c85-9380-87c5531540dc service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:01.557782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 883/1778] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:01.565065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:01.565600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:01.566212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:01.683471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] MFA Rules not processed for user `ef3177a8ead542f1800433cf7b377212`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:01.684734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:01.729616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:01.742059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:01.775765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 896/1779] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:01.776868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:02.016172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f884-e71b-444f-98e6-5441b2b4aba4 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:02.040784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:02.041862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:02.042175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:02.042639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] PATH_INFO: `/v3/projects/eda43add11df48628365612b41ceaacc/users/3a7fb38eef9f4b0ab3b7ab56718fe20c/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:02.047053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 884/1780] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:02.047053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:02.048573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:02.049539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda43add11df48628365612b41ceaacc, user_id=3a7fb38eef9f4b0ab3b7ab56718fe20c, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:02.057753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:02.081443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1978f-9a36-4987-8eed-0dbeef9819b7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:02.101932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 897/1781] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:01 2022] PUT /identity/v3/projects/eda43add11df48628365612b41ceaacc/users/3a7fb38eef9f4b0ab3b7ab56718fe20c/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:02.101932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:02.269790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25779-c28c-41ff-839b-864d8251cc16 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:02.276491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d94912-1010-44b6-85cf-b772cb56c41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:02.277254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d94912-1010-44b6-85cf-b772cb56c41b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:02.277674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d94912-1010-44b6-85cf-b772cb56c41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:02.278069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d94912-1010-44b6-85cf-b772cb56c41b admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:02.278785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d94912-1010-44b6-85cf-b772cb56c41b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:02.282495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:02.300294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d94912-1010-44b6-85cf-b772cb56c41b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:02.312480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:02.316080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:02.316359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:02.316648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] PATH_INFO: `/v3/users/2e33b36670fd46528f6225894c6ff322` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:02.319928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e33b36670fd46528f6225894c6ff322)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:02.329121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:02.404402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 898/1782] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:02 2022] POST /identity/v3/projects => generated 385 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:02.409975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7030-fbbc-43b1-84ce-2829200921ff None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:02.410290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7030-fbbc-43b1-84ce-2829200921ff None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:02.412905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b7030-fbbc-43b1-84ce-2829200921ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:02.445328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e33b36670fd46528f6225894c6ff322'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:02.471577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e33b36670fd46528f6225894c6ff322'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:02.493109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e33b36670fd46528f6225894c6ff322'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:02.493828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e33b36670fd46528f6225894c6ff322'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:02.500587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:02.534473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e33b36670fd46528f6225894c6ff322'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:02.534714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-012b7030-fbbc-43b1-84ce-2829200921ff None None] MFA Rules not processed for user `3a7fb38eef9f4b0ab3b7ab56718fe20c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:02.548757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-012b7030-fbbc-43b1-84ce-2829200921ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:02.558215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 885/1783] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:02 2022] DELETE /identity/v3/users/2e33b36670fd46528f6225894c6ff322 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:02.561050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:02.564910 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:02.697004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 899/1784] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:02.697004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdc2bf-023a-47cb-a699-83e53c5841fb None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:02.697004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdc2bf-023a-47cb-a699-83e53c5841fb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:02.697004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdc2bf-023a-47cb-a699-83e53c5841fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:02.725796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f7fdc2bf-023a-47cb-a699-83e53c5841fb None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:02.727131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f7fdc2bf-023a-47cb-a699-83e53c5841fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:02.766906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d4320-b1a6-4556-910c-9577e1e39233 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:02.782799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:02.783620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:02.783788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:02.784307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:02.785012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:02.792038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:02.851401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 886/1785] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:02 2022] POST /identity/v3/users => generated 299 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:02.856399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:02.860533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:02.920429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 900/1786] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:02.920429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdc2bf-023a-47cb-a699-83e53c5841fb None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:02.958214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdc2bf-023a-47cb-a699-83e53c5841fb None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:02.988344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:02.988979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:02.989912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:02.989912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:02.990337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:02.991981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:03.052569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84390f96-ed38-4160-a9b8-7fd2ef09cf53 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:03.053133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 901/1787] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:03.053425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:03.057491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:03.059863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:03.060545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:03.060828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:03.061890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:03.108664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:03.115343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:03.140780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:03.170348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 887/1788] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:02 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:03.178418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:03.215612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f231-1b81-49be-a61c-96c4b3bf67f9 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:03.238163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:03.239104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:03.239388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:03.239667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] PATH_INFO: `/v3/projects/d268f21bb06e42af8455dc41bcf25c97/users/26e2b40f4a0846eab6ce4634ca59b3be/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:03.273759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b480489-4e45-4680-971b-21edb5a208ad admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:03.278597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d268f21bb06e42af8455dc41bcf25c97, user_id=26e2b40f4a0846eab6ce4634ca59b3be, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:03.283379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:03.289282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:03.290011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:03.290873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:03.291154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:03.328636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 888/1789] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:03 2022] PUT /identity/v3/projects/d268f21bb06e42af8455dc41bcf25c97/users/26e2b40f4a0846eab6ce4634ca59b3be/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:03.335088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:03.346045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:03.350933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:03.392766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 902/1790] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:03.576856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e912-71a3-4f2c-b8bf-f9dd4fd92a68 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:03.595050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:03.598693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:03.599653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:03.600141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:03.601165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:03.604132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:03.635343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 889/1791] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:03.639477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:03.654356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:03.657942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959696-57ba-4c18-af95-27fcc21988f2 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:03.666244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd62bd6-e931-4c12-b399-f5f3a566cd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:03.668738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd62bd6-e931-4c12-b399-f5f3a566cd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:03.669495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd62bd6-e931-4c12-b399-f5f3a566cd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:03.670196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd62bd6-e931-4c12-b399-f5f3a566cd3b admin admin] PATH_INFO: `/v3/projects/d268f21bb06e42af8455dc41bcf25c97/users/26e2b40f4a0846eab6ce4634ca59b3be/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:03.672721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd62bd6-e931-4c12-b399-f5f3a566cd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d268f21bb06e42af8455dc41bcf25c97, user_id=26e2b40f4a0846eab6ce4634ca59b3be, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:03.677097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd62bd6-e931-4c12-b399-f5f3a566cd3b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:03.708677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 903/1792] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:03 2022] PUT /identity/v3/projects/d268f21bb06e42af8455dc41bcf25c97/users/26e2b40f4a0846eab6ce4634ca59b3be/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:03.710510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:03.710938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:03.711691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:03.862318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] MFA Rules not processed for user `26e2b40f4a0846eab6ce4634ca59b3be`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:03.863730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:03.883349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:03.916178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795338b-888e-4895-b28d-d434faf2cf63 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:04.098592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:04.100712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:04.100712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:04.102417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:04.118821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:04.148910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 904/1793] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:04.294554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:04.296074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:04.323703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:04.333552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a0468-5c86-4364-bc94-106d3ebf38b5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:04.358730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30047675-2ef9-4e2d-9dee-f65feb55a598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:04.360092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047675-2ef9-4e2d-9dee-f65feb55a598 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:04.360092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047675-2ef9-4e2d-9dee-f65feb55a598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:04.360092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30047675-2ef9-4e2d-9dee-f65feb55a598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:04.407949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047675-2ef9-4e2d-9dee-f65feb55a598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:04.421799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30047675-2ef9-4e2d-9dee-f65feb55a598 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:04.422535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:04.422535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:04.464521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 890/1794] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:03 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 810 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:04.465657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:04.468038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 905/1795] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:04.473281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:04.613007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32d77c-381f-4e67-9842-80b2bd5e9c12 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:04.635930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:04.636806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:04.637102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:04.637384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] PATH_INFO: `/v3/projects/e9ff424428684c538a9402e9bfca41cc` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:04.638803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ff424428684c538a9402e9bfca41cc)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:04.640783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:04.814147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ff424428684c538a9402e9bfca41cc', 'request_id': 'req-97dde907-6de6-4c5d-b10e-ece27a86f2e6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:04.837570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ff424428684c538a9402e9bfca41cc', 'request_id': 'req-97dde907-6de6-4c5d-b10e-ece27a86f2e6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:04.848039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ff424428684c538a9402e9bfca41cc', 'request_id': 'req-97dde907-6de6-4c5d-b10e-ece27a86f2e6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:04.870975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] The token cache is being invalidate because project e9ff424428684c538a9402e9bfca41cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:04.871270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:04.872065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-97dde907-6de6-4c5d-b10e-ece27a86f2e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:04.877180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 891/1796] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:04 2022] DELETE /identity/v3/projects/e9ff424428684c538a9402e9bfca41cc => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:04.878824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d435-255e-4c18-aeaf-693ac74feae6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:04.879312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d435-255e-4c18-aeaf-693ac74feae6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:04.879312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657d435-255e-4c18-aeaf-693ac74feae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:04.918272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9657d435-255e-4c18-aeaf-693ac74feae6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:04.929906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9657d435-255e-4c18-aeaf-693ac74feae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:04.950269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb95cd3-6b29-4d6c-b7cb-4880c25befdb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:04.950554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb95cd3-6b29-4d6c-b7cb-4880c25befdb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:04.950834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb95cd3-6b29-4d6c-b7cb-4880c25befdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.024692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-eeb95cd3-6b29-4d6c-b7cb-4880c25befdb None None] MFA Rules not processed for user `3a0f25c7ce194ddbb21d1c8b97d03c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:05.031907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eeb95cd3-6b29-4d6c-b7cb-4880c25befdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:05.200647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 906/1797] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:05.202557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441e1c-1ea3-47b6-a65b-7c44bca44fc4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.203961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441e1c-1ea3-47b6-a65b-7c44bca44fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.204568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441e1c-1ea3-47b6-a65b-7c44bca44fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.245936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 892/1798] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:05.267282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f8441e1c-1ea3-47b6-a65b-7c44bca44fc4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:05.268641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f8441e1c-1ea3-47b6-a65b-7c44bca44fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:05.289824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb95cd3-6b29-4d6c-b7cb-4880c25befdb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:05.292026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 907/1799] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:05.311650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb95cd3-6b29-4d6c-b7cb-4880c25befdb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:05.342661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:05.343400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.343743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.344036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.344831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:05.345162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:05.346986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:05.401948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8441e1c-1ea3-47b6-a65b-7c44bca44fc4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:05.407658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 893/1800] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:39:05.412245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8441e1c-1ea3-47b6-a65b-7c44bca44fc4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:05.426467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:05.427280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.427566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.427867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.428545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:05.430520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:05.512723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 908/1801] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:05 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:05.518168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:05.528617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:05.531872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c5307-7049-4f43-979d-95ddebbb62ed admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:05.734262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:05.736521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.738425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.738934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.739903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:05.750007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:05.802804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 894/1802] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:05 2022] POST /identity/v3/users => generated 290 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:05.806508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:05.830671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5bf9f-649e-46e6-8114-86de0c853221 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:05.846479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:05.847505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.848403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.849067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.852228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:05.854143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:05.894144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 909/1803] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:05.896344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:05.914909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9eb53-6932-44ec-a6d0-8cf5c3503577 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:05.928624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad37ea-6890-4601-8752-00a5a154ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:05.930678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad37ea-6890-4601-8752-00a5a154ecbf admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:05.930973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad37ea-6890-4601-8752-00a5a154ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:05.931509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad37ea-6890-4601-8752-00a5a154ecbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:05.932287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad37ea-6890-4601-8752-00a5a154ecbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:05.936930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad37ea-6890-4601-8752-00a5a154ecbf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:05.966169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 895/1804] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:05.967678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:05.989497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534838a-ab46-4c2a-adbc-16e4e9337bf8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:06.002284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:06.003266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:06.004716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:06.005008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] PATH_INFO: `/v3/projects/29e1d1a78d4b4555b273e69c61163b41/users/86167042862443eda668b2f5ef2cb015/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:06.053992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e1d1a78d4b4555b273e69c61163b41, user_id=86167042862443eda668b2f5ef2cb015, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:06.057106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:06.116536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 910/1805] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:05 2022] PUT /identity/v3/projects/29e1d1a78d4b4555b273e69c61163b41/users/86167042862443eda668b2f5ef2cb015/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:06.121001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210479e6-7276-4691-85df-42163d7074ac None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:06.121312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210479e6-7276-4691-85df-42163d7074ac None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:06.121597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210479e6-7276-4691-85df-42163d7074ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:06.203726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-210479e6-7276-4691-85df-42163d7074ac None None] MFA Rules not processed for user `86167042862443eda668b2f5ef2cb015`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:06.205005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-210479e6-7276-4691-85df-42163d7074ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:06.323939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:06.325792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:06.396751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:06.405483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 896/1806] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:06.412140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210479e6-7276-4691-85df-42163d7074ac None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:06.545603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb05d48-1015-4f56-bb4d-65763c62503a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:06.560630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210479e6-7276-4691-85df-42163d7074ac None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:06.565436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:06.566930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:06.567305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:06.567601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:06.567885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:06.568482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:06.569162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:06.569635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:06.570626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:06.573022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:06.579897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:06.607192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:06.630345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 897/1807] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:06 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:06.636070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:06.829593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:06.849449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:06.893687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 911/1808] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:06.910604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade6291-5bce-4448-b158-793f83b14a2e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:06.929245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:06.939618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:06.940802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:06.941188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:06.942052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:06.960208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:07.033028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 898/1809] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:06 2022] POST /identity/v3/users => generated 297 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:07.037337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:07.053995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfaea18-e636-46dc-a14c-55a058d3a414 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:07.067718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:07.069286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:07.070374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:07.072166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:07.073229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:07.078872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:07.103528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 912/1810] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:07.107335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:07.139410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d405a4a-d380-4d28-81a9-c83ee0c2ef37 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:07.165795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:07.169948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:07.174143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:07.174638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] PATH_INFO: `/v3/projects/a84751c2bbd747519f1f4807b30738d0/users/6e895c9ae0274f2481aaa253d63c1be1/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:07.236746 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84751c2bbd747519f1f4807b30738d0, user_id=6e895c9ae0274f2481aaa253d63c1be1, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:07.242953 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:07.275335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 899/1811] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:07 2022] PUT /identity/v3/projects/a84751c2bbd747519f1f4807b30738d0/users/6e895c9ae0274f2481aaa253d63c1be1/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:07.279477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:07.298562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab30d76-6c5f-4065-a517-837e49ad59d8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:07.490937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84ea3a-e07c-4630-87c3-c308379069a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:07.491688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84ea3a-e07c-4630-87c3-c308379069a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:07.491983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84ea3a-e07c-4630-87c3-c308379069a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:07.492266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84ea3a-e07c-4630-87c3-c308379069a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:07.493021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84ea3a-e07c-4630-87c3-c308379069a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:07.495798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84ea3a-e07c-4630-87c3-c308379069a3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:07.522469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 913/1812] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:07.524672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:07.562636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c440091-7507-4191-869c-a92f382c310b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:07.588099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:07.592316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:07.595723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:07.601248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] PATH_INFO: `/v3/projects/a84751c2bbd747519f1f4807b30738d0/users/6e895c9ae0274f2481aaa253d63c1be1/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:07.617265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84751c2bbd747519f1f4807b30738d0, user_id=6e895c9ae0274f2481aaa253d63c1be1, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:07.626972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:07.684192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 900/1813] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:07 2022] PUT /identity/v3/projects/a84751c2bbd747519f1f4807b30738d0/users/6e895c9ae0274f2481aaa253d63c1be1/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:07.696395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:07.696956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:07.697317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:07.803571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] MFA Rules not processed for user `6e895c9ae0274f2481aaa253d63c1be1`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:07.810266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:08.040266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:08.088294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 914/1814] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:08.106756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:08.108702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:08.248114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:08.249694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:08.292184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b3824-43e1-405c-9624-ade764d81e0d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:08.329914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f4afe-a539-400d-9b1b-104a8c10ad34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:08.329914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f4afe-a539-400d-9b1b-104a8c10ad34 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:08.329914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f4afe-a539-400d-9b1b-104a8c10ad34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:08.329914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f4afe-a539-400d-9b1b-104a8c10ad34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:08.367585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f4afe-a539-400d-9b1b-104a8c10ad34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:08.372525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f4afe-a539-400d-9b1b-104a8c10ad34 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:08.451032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f28c8-fcaf-401b-8791-16669e3053e0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:08.453157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 901/1815] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:08.467821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:08.468796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:08.469086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:08.469363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] PATH_INFO: `/v3/users/f20ce154d4da4e7e8719b31ae0382b27` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:08.471682 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f20ce154d4da4e7e8719b31ae0382b27)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:08.476223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:08.480180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064835ea-7e2f-4600-b11c-1b5c21d2543c None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:08.480740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064835ea-7e2f-4600-b11c-1b5c21d2543c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:08.481233 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064835ea-7e2f-4600-b11c-1b5c21d2543c None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:08.482904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 902/1816] 10.222.0.25 () {54 vars in 912 bytes} [Fri Jun 17 11:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 17 11:39:08.498053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:08.498671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:08.499313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:08.563589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] MFA Rules not processed for user `b3b35d6d2cd848f4841352fbb58f2053`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:08.564904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:08.643747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f20ce154d4da4e7e8719b31ae0382b27'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:08.672012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f20ce154d4da4e7e8719b31ae0382b27'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:08.686471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f20ce154d4da4e7e8719b31ae0382b27'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:08.687676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f20ce154d4da4e7e8719b31ae0382b27'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:08.713064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 903/1817] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:39:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:08.714001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:08.717516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:08.726489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:08.733057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3f571624-22c6-4a9b-bc82-56e2533d67ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f20ce154d4da4e7e8719b31ae0382b27'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:08.746407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 915/1818] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:08 2022] DELETE /identity/v3/users/f20ce154d4da4e7e8719b31ae0382b27 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:08.750568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4ecb-040b-41a0-b48f-d2dab8296c32 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:08.752769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:08.753059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:08.753354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:08.826447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:08.829587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:08.925805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:08.927790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:08.928977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:08.929463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:08.942378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:08.990717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 916/1819] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:08.991520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:08.992894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:09.006242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:09.076650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:09.080398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:09.105803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed0778-6dd0-46db-921b-2aad9068f3db None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:09.108469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 904/1820] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:09.109443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:09.113097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:09.119574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433630b5-c2ce-4a60-a030-972192803808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:09.120262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433630b5-c2ce-4a60-a030-972192803808 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:09.120546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433630b5-c2ce-4a60-a030-972192803808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:09.120825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433630b5-c2ce-4a60-a030-972192803808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:09.139724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433630b5-c2ce-4a60-a030-972192803808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:09.140785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf3e07-8fa9-4f77-8993-f39ed8fd9600 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:09.147949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6fc84-b7a2-4f87-888d-d13a529b2319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:09.148630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6fc84-b7a2-4f87-888d-d13a529b2319 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:09.149730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6fc84-b7a2-4f87-888d-d13a529b2319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:09.150072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6fc84-b7a2-4f87-888d-d13a529b2319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:09.158304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433630b5-c2ce-4a60-a030-972192803808 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:09.180076 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 917/1821] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:09.185780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6fc84-b7a2-4f87-888d-d13a529b2319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:09.187908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6fc84-b7a2-4f87-888d-d13a529b2319 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:09.206632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 905/1822] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:09.613906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdf95-4bf2-45f4-92ba-578e98562bcc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:09.614308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdf95-4bf2-45f4-92ba-578e98562bcc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:09.614616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdf95-4bf2-45f4-92ba-578e98562bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:09.658484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:09.659230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:09.659449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:09.688928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-672bdf95-4bf2-45f4-92ba-578e98562bcc None None] MFA Rules not processed for user `c2c930cfab47464fb784c98f58abbb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:09.690228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-672bdf95-4bf2-45f4-92ba-578e98562bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:09.719625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] MFA Rules not processed for user `9e209f3667f54c2b9bc6eab13010296c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:09.725527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:09.886138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 906/1823] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:09.887437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:09.893990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:09.894764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 918/1824] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:09.922645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a402b-b367-4e64-92f4-f7bd3fd2524e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:09.936004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:09.937229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:09.937665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:09.937971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] PATH_INFO: `/v3/projects/50cb4db522f24a3e8dd7efc51499db96` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:09.942140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cb4db522f24a3e8dd7efc51499db96)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:09.945288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:10.033230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:10.036621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cb4db522f24a3e8dd7efc51499db96', 'request_id': 'req-fc08f49e-ae66-414a-a128-aa37c8a38e31'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.051033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cb4db522f24a3e8dd7efc51499db96', 'request_id': 'req-fc08f49e-ae66-414a-a128-aa37c8a38e31'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.067976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cb4db522f24a3e8dd7efc51499db96', 'request_id': 'req-fc08f49e-ae66-414a-a128-aa37c8a38e31'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.092930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] The token cache is being invalidate because project 50cb4db522f24a3e8dd7efc51499db96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:10.093225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.097012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.101063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 907/1825] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:09 2022] DELETE /identity/v3/projects/50cb4db522f24a3e8dd7efc51499db96 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:10.105108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bdf95-4bf2-45f4-92ba-578e98562bcc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:10.125568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bdf95-4bf2-45f4-92ba-578e98562bcc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:10.272954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:10.274353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:10.274643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:10.275050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] PATH_INFO: `/v3/users/39f630dcc1b04dd983537f128e5bc7df` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:10.278302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f630dcc1b04dd983537f128e5bc7df)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:10.281204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:10.357970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f630dcc1b04dd983537f128e5bc7df'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.379375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f630dcc1b04dd983537f128e5bc7df'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.390158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:10.403704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f630dcc1b04dd983537f128e5bc7df'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.404734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f630dcc1b04dd983537f128e5bc7df'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.427594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1fdadf4f-41be-439c-bd4b-28c2c9805291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f630dcc1b04dd983537f128e5bc7df'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:10.473727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 919/1826] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:10 2022] DELETE /identity/v3/users/39f630dcc1b04dd983537f128e5bc7df => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:11.137484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:11.139670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:11.159715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08f49e-ae66-414a-a128-aa37c8a38e31 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:11.284886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:11.285703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:11.289884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:11.289884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] PATH_INFO: `/v3/projects/619e67c837494938949f78d238ddd962` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:11.290935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=619e67c837494938949f78d238ddd962)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:11.293762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:11.398620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '619e67c837494938949f78d238ddd962', 'request_id': 'req-237ef824-a76b-4000-939f-eaa4dcfe52b6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:11.407548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '619e67c837494938949f78d238ddd962', 'request_id': 'req-237ef824-a76b-4000-939f-eaa4dcfe52b6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:11.414718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '619e67c837494938949f78d238ddd962', 'request_id': 'req-237ef824-a76b-4000-939f-eaa4dcfe52b6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:11.434054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] The token cache is being invalidate because project 619e67c837494938949f78d238ddd962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:11.434521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:11.435525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-237ef824-a76b-4000-939f-eaa4dcfe52b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:11.439385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 908/1827] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:11 2022] DELETE /identity/v3/projects/619e67c837494938949f78d238ddd962 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:11.445136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035103d-f672-4160-b3ca-2abbcd279ef8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:11.445618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035103d-f672-4160-b3ca-2abbcd279ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:11.445742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035103d-f672-4160-b3ca-2abbcd279ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:11.496958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9035103d-f672-4160-b3ca-2abbcd279ef8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:11.498121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9035103d-f672-4160-b3ca-2abbcd279ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:11.642137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 920/1828] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:11.649361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d265dbc-8d25-4392-8606-d2828beb9f65 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:11.649659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d265dbc-8d25-4392-8606-d2828beb9f65 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:11.649967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d265dbc-8d25-4392-8606-d2828beb9f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:11.690228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9d265dbc-8d25-4392-8606-d2828beb9f65 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:11.692171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9d265dbc-8d25-4392-8606-d2828beb9f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:11.719067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 909/1829] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:11.726184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035103d-f672-4160-b3ca-2abbcd279ef8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:11.742283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035103d-f672-4160-b3ca-2abbcd279ef8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:11.764690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:11.765798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:11.766322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:11.767022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:11.769038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:11.770235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:11.772385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:11.813966 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 921/1830] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:11.819012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d265dbc-8d25-4392-8606-d2828beb9f65 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:11.832829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d265dbc-8d25-4392-8606-d2828beb9f65 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:11.847133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:11.849674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:11.850267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:11.851060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:11.875706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:11.882370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.005451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 910/1831] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:11 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:12.013961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.030005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d393621-da6d-4b88-92a3-34ee3dd6c3dd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.129173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.130066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.130515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.130807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.131525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.134612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.187355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 922/1832] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:12 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:12.191111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.207311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82d17-f9ec-4db0-b832-0ec25d4ba5cf admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.226798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.228160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.229656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.229831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.231919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.233991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.251907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 911/1833] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:12.256079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.270551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82461d97-e287-4e95-bcbd-47464e9c27e6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.282538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1fcac-5559-41f7-b815-b364d022a38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.283384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1fcac-5559-41f7-b815-b364d022a38f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.283680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1fcac-5559-41f7-b815-b364d022a38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.284220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1fcac-5559-41f7-b815-b364d022a38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.284963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1fcac-5559-41f7-b815-b364d022a38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.286604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1fcac-5559-41f7-b815-b364d022a38f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.306114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 923/1834] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:12.309945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.331107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b01bb-e876-4fe6-97b1-5e8531cb4cd3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.336629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.339711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.340378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.340838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] PATH_INFO: `/v3/projects/ee76bb8d4b804bd794696fdecc929e6d/users/8c5f33ccd44b4e03af3dfe2920006c83/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.360046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee76bb8d4b804bd794696fdecc929e6d, user_id=8c5f33ccd44b4e03af3dfe2920006c83, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.362871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.372289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:12.380758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 912/1835] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:12 2022] PUT /identity/v3/projects/ee76bb8d4b804bd794696fdecc929e6d/users/8c5f33ccd44b4e03af3dfe2920006c83/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:12.393338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac625d-f367-49cc-9753-7f4a94864eda None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.393661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac625d-f367-49cc-9753-7f4a94864eda None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.393961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac625d-f367-49cc-9753-7f4a94864eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.427358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-25ac625d-f367-49cc-9753-7f4a94864eda None None] MFA Rules not processed for user `8c5f33ccd44b4e03af3dfe2920006c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:12.428611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-25ac625d-f367-49cc-9753-7f4a94864eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:12.468490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:12.514104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 924/1836] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:12.518976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.583039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3283c31-5a01-4e3e-8267-7d637e14c441 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.588255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.588918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.589208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.589483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.590218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.592360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.618786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 913/1837] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:12 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:12.622550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac625d-f367-49cc-9753-7f4a94864eda None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.634316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac625d-f367-49cc-9753-7f4a94864eda None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.693710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.694425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.694714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.694992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.695696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.698162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.719094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 925/1838] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:12 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:12.723132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.735263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bbc8f-7012-41d1-933b-0ba8b0d7ae1d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.740891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.741572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.741898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.742171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.742934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.745869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.756806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 914/1839] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:12.762010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.773766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d150f-27f4-46c7-a15e-28157f7dfce9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.781391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c4657-df34-49d3-bd0a-820303f2ea2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.782087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c4657-df34-49d3-bd0a-820303f2ea2d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.782373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c4657-df34-49d3-bd0a-820303f2ea2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.782681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c4657-df34-49d3-bd0a-820303f2ea2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.783520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c4657-df34-49d3-bd0a-820303f2ea2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.785249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c4657-df34-49d3-bd0a-820303f2ea2d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.798025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 926/1840] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:12.800856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:12.811568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a4073-5a4a-445a-b805-8699a937f031 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:12.817260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:12.818294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.818648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.818997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] PATH_INFO: `/v3/projects/2b4f2e7848894ed798faae6e4e521d62/users/600f6c6b83ce40849f61576b195b2b9b/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.835155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4f2e7848894ed798faae6e4e521d62, user_id=600f6c6b83ce40849f61576b195b2b9b, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:12.838576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:12.861146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 915/1841] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:12 2022] PUT /identity/v3/projects/2b4f2e7848894ed798faae6e4e521d62/users/600f6c6b83ce40849f61576b195b2b9b/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:12.870852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a853c2-e702-43fd-8315-2050a1ed88d7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:12.871222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a853c2-e702-43fd-8315-2050a1ed88d7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:12.871509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a853c2-e702-43fd-8315-2050a1ed88d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:12.932400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c2a853c2-e702-43fd-8315-2050a1ed88d7 None None] MFA Rules not processed for user `600f6c6b83ce40849f61576b195b2b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:12.933607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c2a853c2-e702-43fd-8315-2050a1ed88d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:13.141217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 927/1842] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:13.143235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:13.216738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0359f2d-5bbe-4e22-b624-63468feed8fb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:13.221684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:13.222397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.222685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.222964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.223663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:13.225602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:13.257817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 916/1843] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:13 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:13.261827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a853c2-e702-43fd-8315-2050a1ed88d7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:13.273570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a853c2-e702-43fd-8315-2050a1ed88d7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:13.365145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:13.394840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:13.395705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.396200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.396668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.397609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:13.399986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:13.422785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 928/1844] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:13 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:13.428016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:13.446539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330ab39-3cf1-4497-a06d-4888b78c6d8b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:13.462884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:13.463765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.464193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.465478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.465478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:13.467226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:13.493812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 917/1845] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:13.498526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:13.509969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85483f-fc5b-42fa-af8c-f90002fb8257 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:13.516588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:13.517634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.518055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.518228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] PATH_INFO: `/v3/projects/f9dc28645dee4e5a92d59b3e994ed7c0/users/6efac012ba0c4da08ad6456c839c04b4/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.540409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9dc28645dee4e5a92d59b3e994ed7c0, user_id=6efac012ba0c4da08ad6456c839c04b4, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:13.543351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:13.565943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 929/1846] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:13 2022] PUT /identity/v3/projects/f9dc28645dee4e5a92d59b3e994ed7c0/users/6efac012ba0c4da08ad6456c839c04b4/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:13.568794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:13.596518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29584c58-0a06-45f6-bfd3-efd4f10d0326 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:13.621311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:13.668180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9df9cd-aea8-4cc2-9a6c-ee0ebc77718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:13.669797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9df9cd-aea8-4cc2-9a6c-ee0ebc77718b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.670454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9df9cd-aea8-4cc2-9a6c-ee0ebc77718b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.670957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9df9cd-aea8-4cc2-9a6c-ee0ebc77718b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.672064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9df9cd-aea8-4cc2-9a6c-ee0ebc77718b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:13.674458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9df9cd-aea8-4cc2-9a6c-ee0ebc77718b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:13.685899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 918/1847] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:13.689929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:13.700821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4aaa1-a95c-4cfc-b424-105d3800ed86 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:13.705581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:13.706534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.706819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.707092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] PATH_INFO: `/v3/projects/f9dc28645dee4e5a92d59b3e994ed7c0/users/6efac012ba0c4da08ad6456c839c04b4/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.709355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9dc28645dee4e5a92d59b3e994ed7c0, user_id=6efac012ba0c4da08ad6456c839c04b4, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:13.712583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:13.723366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 930/1848] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:13 2022] PUT /identity/v3/projects/f9dc28645dee4e5a92d59b3e994ed7c0/users/6efac012ba0c4da08ad6456c839c04b4/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:13.734468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5a5f-11d8-4d96-bd7a-c16af9d97f3b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.734760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5a5f-11d8-4d96-bd7a-c16af9d97f3b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.735056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5a5f-11d8-4d96-bd7a-c16af9d97f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.763275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-32db5a5f-11d8-4d96-bd7a-c16af9d97f3b None None] MFA Rules not processed for user `6efac012ba0c4da08ad6456c839c04b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:13.764483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-32db5a5f-11d8-4d96-bd7a-c16af9d97f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:13.850962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 919/1849] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:13.862763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:13.864697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:13.926655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67dd0d-5d22-485f-9c41-b774ef15ee63 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:13.930283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:13.931009 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:13.931428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:13.931565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:13.941824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:13.943588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:13.955111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 931/1850] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:14.003663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5a5f-11d8-4d96-bd7a-c16af9d97f3b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:14.021881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5a5f-11d8-4d96-bd7a-c16af9d97f3b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:14.027390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:14.028764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:14.029056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:14.029377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:14.104341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:14.105787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:14.116511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 920/1851] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:14.223240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:14.224760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:14.236261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce47c-b138-4fa5-9bb8-e9e08a156ba4 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:14.240471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cb5b1-a25c-4065-bba9-7872b2fd9b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:14.241151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cb5b1-a25c-4065-bba9-7872b2fd9b10 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:14.241435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cb5b1-a25c-4065-bba9-7872b2fd9b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:14.241707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cb5b1-a25c-4065-bba9-7872b2fd9b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:14.243590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9e3cb5b1-a25c-4065-bba9-7872b2fd9b10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:14.307479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cb5b1-a25c-4065-bba9-7872b2fd9b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:14.308953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cb5b1-a25c-4065-bba9-7872b2fd9b10 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:14.318313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 932/1852] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:14.773931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:14.775631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:14.823575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:14.842800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a308b-4011-4d68-b737-1b050bceaa7f service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:14.847324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:14.848206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:14.848491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:14.848770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] PATH_INFO: `/v3/users/23d3e173da5b4430a47adcc76d025261` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:14.849988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d3e173da5b4430a47adcc76d025261)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:14.851942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:14.898335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d3e173da5b4430a47adcc76d025261'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:14.905137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d3e173da5b4430a47adcc76d025261'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:14.912120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d3e173da5b4430a47adcc76d025261'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:14.912918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d3e173da5b4430a47adcc76d025261'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:14.925643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d3e173da5b4430a47adcc76d025261'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:14.932394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 921/1853] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:14 2022] DELETE /identity/v3/users/23d3e173da5b4430a47adcc76d025261 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:14.936932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:14.937376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:14.937484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:14.954040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:14.955240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:14.983758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:15.015671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 933/1854] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:15.025343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:15.027139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:15.216842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7590b-6211-4a72-a95e-5200ffa279f5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:15.221182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:15.221879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:15.222173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:15.222477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:15.232964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:15.234517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:15.244335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 922/1855] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:15.522994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:15.525018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:15.556372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e840548-b117-4063-b90f-fd4297a95ee5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:15.561006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:15.561905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:15.562223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:15.562502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] PATH_INFO: `/v3/projects/3f4ecfc760014c57a09e0e5a5c769e04` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:15.563720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f4ecfc760014c57a09e0e5a5c769e04)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:15.565624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:15.613048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f4ecfc760014c57a09e0e5a5c769e04', 'request_id': 'req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.620250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f4ecfc760014c57a09e0e5a5c769e04', 'request_id': 'req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.629419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f4ecfc760014c57a09e0e5a5c769e04', 'request_id': 'req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.641356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] The token cache is being invalidate because project 3f4ecfc760014c57a09e0e5a5c769e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:15.641648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.642323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.643854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 934/1856] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:15 2022] DELETE /identity/v3/projects/3f4ecfc760014c57a09e0e5a5c769e04 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:15.647795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:15.657774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633253bd-84ac-42b7-8776-dd27b7e7648d service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:15.712533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:15.713426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:15.713706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:15.714013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] PATH_INFO: `/v3/users/3a7fb38eef9f4b0ab3b7ab56718fe20c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:15.715191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7fb38eef9f4b0ab3b7ab56718fe20c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:15.717154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:15.721542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:15.766006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7fb38eef9f4b0ab3b7ab56718fe20c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.773112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7fb38eef9f4b0ab3b7ab56718fe20c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.781146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7fb38eef9f4b0ab3b7ab56718fe20c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.781773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7fb38eef9f4b0ab3b7ab56718fe20c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.795761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f929b6bf-38f3-4669-8d84-64c589945a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7fb38eef9f4b0ab3b7ab56718fe20c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:15.803452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 923/1857] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:15 2022] DELETE /identity/v3/users/3a7fb38eef9f4b0ab3b7ab56718fe20c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:16.106315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.107868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:16.122236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9bfc0-3fcf-43e0-be29-a433a14ca58a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.182370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.183231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.183518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.183792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] PATH_INFO: `/v3/projects/eda43add11df48628365612b41ceaacc` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.184973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda43add11df48628365612b41ceaacc)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.186924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:16.241199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda43add11df48628365612b41ceaacc', 'request_id': 'req-26b71742-dedb-4c89-8635-df300482d361'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:16.290555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda43add11df48628365612b41ceaacc', 'request_id': 'req-26b71742-dedb-4c89-8635-df300482d361'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:16.294614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:16.298529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda43add11df48628365612b41ceaacc', 'request_id': 'req-26b71742-dedb-4c89-8635-df300482d361'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:16.310378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] The token cache is being invalidate because project eda43add11df48628365612b41ceaacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:16.315794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:16.315794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-26b71742-dedb-4c89-8635-df300482d361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:16.315794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 935/1858] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:16 2022] DELETE /identity/v3/projects/eda43add11df48628365612b41ceaacc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:16.321698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d1638-c38a-411a-bae8-31e733b5b4bd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.321698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d1638-c38a-411a-bae8-31e733b5b4bd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.321698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d1638-c38a-411a-bae8-31e733b5b4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.343559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a95d1638-c38a-411a-bae8-31e733b5b4bd None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:16.344316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a95d1638-c38a-411a-bae8-31e733b5b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:16.441996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 924/1859] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:16.449861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb32e-f2a8-464d-a69b-0675d9b092d3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.450167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb32e-f2a8-464d-a69b-0675d9b092d3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.450451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb32e-f2a8-464d-a69b-0675d9b092d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.468753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ed8eb32e-f2a8-464d-a69b-0675d9b092d3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:16.470008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ed8eb32e-f2a8-464d-a69b-0675d9b092d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:16.478985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 936/1860] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:16.490284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d1638-c38a-411a-bae8-31e733b5b4bd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.501932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d1638-c38a-411a-bae8-31e733b5b4bd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.508345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.508849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.509135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.509415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.510236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:16.510558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.512218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:16.529503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 925/1861] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:16.530182 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb32e-f2a8-464d-a69b-0675d9b092d3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.539711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb32e-f2a8-464d-a69b-0675d9b092d3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.544838 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.545495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.545773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.546076 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.546784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.548671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:16.578579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 937/1862] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:16 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:16.582423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.593187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1c6b-3b94-4407-bd48-9990208fc7b1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.652391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.653016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.653304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.653584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.654318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.656246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:16.672092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:16.678166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 926/1863] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:16 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:16.682224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.692379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bb203-1ced-475a-97f0-171dde9ac232 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.696914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.697615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.697925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.698208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.698933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.700525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:16.710843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 938/1864] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:16.715098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.724832 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fba054-0695-4cdf-9476-15d1db8a0686 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.729951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3af11d-715b-422f-97f9-c677ccca697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.730603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3af11d-715b-422f-97f9-c677ccca697b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.730884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3af11d-715b-422f-97f9-c677ccca697b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.731153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3af11d-715b-422f-97f9-c677ccca697b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.731867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3af11d-715b-422f-97f9-c677ccca697b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.733452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3af11d-715b-422f-97f9-c677ccca697b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:16.743132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 927/1865] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:16.747054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.756804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bd4ab-85d5-4d08-a623-b9f39d20d2d8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.761430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.762335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.762617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.762890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] PATH_INFO: `/v3/projects/994b68ab31514facb77713ed4c74fe97/users/068fa4f424cd470288bb8de924373151/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.777358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=994b68ab31514facb77713ed4c74fe97, user_id=068fa4f424cd470288bb8de924373151, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.780000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:16.793651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 939/1866] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:16 2022] PUT /identity/v3/projects/994b68ab31514facb77713ed4c74fe97/users/068fa4f424cd470288bb8de924373151/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:16.811281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f272c-db92-4220-a1d3-86e1315f0c3b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.811821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f272c-db92-4220-a1d3-86e1315f0c3b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.812034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f272c-db92-4220-a1d3-86e1315f0c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.848862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-330f272c-db92-4220-a1d3-86e1315f0c3b None None] MFA Rules not processed for user `068fa4f424cd470288bb8de924373151`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:16.850115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-330f272c-db92-4220-a1d3-86e1315f0c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:16.919797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 928/1867] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:16.924729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:16.988829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7d527-fd6e-412a-84b6-6d80388cd4f7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:16.993646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:16.995071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:16.995449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:16.995697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:16.996388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:16.998382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:17.026790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 940/1868] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:16 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:17.028003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f272c-db92-4220-a1d3-86e1315f0c3b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:17.039243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f272c-db92-4220-a1d3-86e1315f0c3b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:17.099199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:17.099974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:17.100301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:17.100600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:17.101387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:17.104024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:17.127687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 929/1869] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:17 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:17.134713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:17.149263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bc588-9b8e-4b35-a03e-8314f83c4b33 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:17.154661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:17.155483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:17.155798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:17.156070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:17.156812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:17.158465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:17.171064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 941/1870] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:17.175274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:17.186403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c890c22-fffb-4e16-873d-292cedccdd8d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:17.194704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:17.198017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:17.200691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:17.201048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] PATH_INFO: `/v3/projects/4fc651f0622a4a38b0e8aadb76b259be/users/cf48104be3054ee89a9ee2e0afc3314f/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:17.217081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc651f0622a4a38b0e8aadb76b259be, user_id=cf48104be3054ee89a9ee2e0afc3314f, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:17.219889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:17.234337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 930/1871] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:17 2022] PUT /identity/v3/projects/4fc651f0622a4a38b0e8aadb76b259be/users/cf48104be3054ee89a9ee2e0afc3314f/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:17.238570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:17.253800 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc912fc4-6cb9-41a1-b6bd-f400a9b0be5b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:17.314065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:17.330152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0e4ec-930a-4745-9430-1657d2a172d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:17.330152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0e4ec-930a-4745-9430-1657d2a172d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:17.330152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0e4ec-930a-4745-9430-1657d2a172d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:17.330152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0e4ec-930a-4745-9430-1657d2a172d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:17.330502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0e4ec-930a-4745-9430-1657d2a172d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:17.342677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab0e4ec-930a-4745-9430-1657d2a172d2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:17.365881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 942/1872] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:17.370329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:17.390933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e84ffe-82ae-4ac9-8a78-39c6a616e77b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:17.396868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:17.397935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:17.398203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:17.398651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] PATH_INFO: `/v3/projects/4fc651f0622a4a38b0e8aadb76b259be/users/cf48104be3054ee89a9ee2e0afc3314f/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:17.403253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc651f0622a4a38b0e8aadb76b259be, user_id=cf48104be3054ee89a9ee2e0afc3314f, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:17.406129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:17.421391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 931/1873] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:17 2022] PUT /identity/v3/projects/4fc651f0622a4a38b0e8aadb76b259be/users/cf48104be3054ee89a9ee2e0afc3314f/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:17.432843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c07cfa-b0c3-4238-8f5c-0cef1b639ec9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:17.433164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c07cfa-b0c3-4238-8f5c-0cef1b639ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:17.433376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c07cfa-b0c3-4238-8f5c-0cef1b639ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:17.463543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-08c07cfa-b0c3-4238-8f5c-0cef1b639ec9 None None] MFA Rules not processed for user `cf48104be3054ee89a9ee2e0afc3314f`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:17.465031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08c07cfa-b0c3-4238-8f5c-0cef1b639ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:17.576278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 943/1874] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:17.593533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:17.595325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:17.646948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:17.728664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea099-07fd-41db-9a4f-09361003c6dc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:17.737768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002f5a-b2e3-4da6-be2f-fc2d165e2255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:17.739813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002f5a-b2e3-4da6-be2f-fc2d165e2255 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:17.739813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002f5a-b2e3-4da6-be2f-fc2d165e2255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:17.740065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002f5a-b2e3-4da6-be2f-fc2d165e2255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:17.766382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002f5a-b2e3-4da6-be2f-fc2d165e2255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:17.768105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002f5a-b2e3-4da6-be2f-fc2d165e2255 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:17.805204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 932/1875] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:17.994425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c07cfa-b0c3-4238-8f5c-0cef1b639ec9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:18.015646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c07cfa-b0c3-4238-8f5c-0cef1b639ec9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:18.025178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7a027-5a35-4b52-b57d-2e877bab768d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:18.026037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7a027-5a35-4b52-b57d-2e877bab768d service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:18.026428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7a027-5a35-4b52-b57d-2e877bab768d service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:18.026804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7a027-5a35-4b52-b57d-2e877bab768d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:18.185789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7a027-5a35-4b52-b57d-2e877bab768d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:18.188335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7a027-5a35-4b52-b57d-2e877bab768d service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:18.208766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 944/1876] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:17 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:19.461606 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515354c-1abe-49a8-8643-15c0a350a7de None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:19.462319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515354c-1abe-49a8-8643-15c0a350a7de None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:19.462319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515354c-1abe-49a8-8643-15c0a350a7de None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:19.464714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 933/1877] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:39:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:19.495008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:19.495558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:19.495639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:19.532307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:19.534504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:19.655079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515354c-1abe-49a8-8643-15c0a350a7de None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:19.656775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e515354c-1abe-49a8-8643-15c0a350a7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:19.663459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 945/1878] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:39:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:19.696608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:19.698752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:19.719696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed83de-be5a-4c43-b1f1-f6446569fff3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:19.729207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:19.730103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:19.730500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:19.730926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:19.741702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:19.747343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:19.749431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:19.766790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 946/1879] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:19.829875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515354c-1abe-49a8-8643-15c0a350a7de None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:19.841488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:19.842551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:19.843073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:19.845100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] PATH_INFO: `/v3/users/8f3f7c77b5204d43afac5996a1d587f8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:19.847163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3f7c77b5204d43afac5996a1d587f8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:19.849558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:19.987555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3f7c77b5204d43afac5996a1d587f8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.003593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3f7c77b5204d43afac5996a1d587f8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.044167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3f7c77b5204d43afac5996a1d587f8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.044959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3f7c77b5204d43afac5996a1d587f8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.052722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:20.054391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:20.062914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-baab4b89-dafd-410f-a1f8-f713bb2627b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3f7c77b5204d43afac5996a1d587f8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.072988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 934/1880] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:19 2022] DELETE /identity/v3/users/8f3f7c77b5204d43afac5996a1d587f8 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:20.076363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:20.076661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:20.076943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:20.086992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9838ded-a61f-48f1-ba83-a4630c59f7fc service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:20.153265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:20.156092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:20.251885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:20.255537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:20.256867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:20.259574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:20.260078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] PATH_INFO: `/v3/users/8c5f33ccd44b4e03af3dfe2920006c83` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:20.262619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5f33ccd44b4e03af3dfe2920006c83)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:20.266030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:20.283557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 935/1881] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:20.376612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5f33ccd44b4e03af3dfe2920006c83'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.392599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5f33ccd44b4e03af3dfe2920006c83'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.405965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5f33ccd44b4e03af3dfe2920006c83'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.407103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5f33ccd44b4e03af3dfe2920006c83'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.423403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:20.426142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:20.483105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-34238a87-27aa-4e40-a6a8-37e9eca52513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5f33ccd44b4e03af3dfe2920006c83'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:20.501125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 947/1882] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:20 2022] DELETE /identity/v3/users/8c5f33ccd44b4e03af3dfe2920006c83 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:20.505475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:20.506205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:20.506488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:20.531692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:20.534611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:20.609827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab40fd-bbc0-48a2-a0ad-ee138225f6f4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:20.666483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 948/1883] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:20.710840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c553b6d-b532-47f7-b4da-969b2ec0784f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:20.711800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553b6d-b532-47f7-b4da-969b2ec0784f service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:20.712097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553b6d-b532-47f7-b4da-969b2ec0784f service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:20.712623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c553b6d-b532-47f7-b4da-969b2ec0784f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:20.715295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2c553b6d-b532-47f7-b4da-969b2ec0784f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:20.734619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c553b6d-b532-47f7-b4da-969b2ec0784f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:20.736229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c553b6d-b532-47f7-b4da-969b2ec0784f service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:20.747835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 936/1884] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:20.759203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:20.761473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:20.779648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fd5bc-158a-4992-864e-d61d0bc63d9b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:20.801732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55724321-94b2-4817-9b19-6335ee0ab816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:20.802514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55724321-94b2-4817-9b19-6335ee0ab816 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:20.802868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55724321-94b2-4817-9b19-6335ee0ab816 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:20.803810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55724321-94b2-4817-9b19-6335ee0ab816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:20.832827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55724321-94b2-4817-9b19-6335ee0ab816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:20.834560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55724321-94b2-4817-9b19-6335ee0ab816 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:20.848155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 949/1885] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:21.919685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ec86f-94ab-4ed0-8283-a67bffb0b5cd None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:21.920605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ec86f-94ab-4ed0-8283-a67bffb0b5cd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:21.921113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ec86f-94ab-4ed0-8283-a67bffb0b5cd None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:21.922655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 937/1886] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:39:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:39:21.987292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:21.988279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:21.989905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:21.994903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ec86f-94ab-4ed0-8283-a67bffb0b5cd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:21.996884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c19ec86f-94ab-4ed0-8283-a67bffb0b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.004930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:22.036054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ec86f-94ab-4ed0-8283-a67bffb0b5cd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:22.042023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:22.043131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:22.043585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:22.044101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] PATH_INFO: `/v3/projects/0af8188ddb76467da2ed1183f127d5f8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:22.051768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af8188ddb76467da2ed1183f127d5f8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:22.053978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:22.056699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:22.058377 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.179563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af8188ddb76467da2ed1183f127d5f8', 'request_id': 'req-10cba0d2-d245-400c-9d60-958c369cea69'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.204619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af8188ddb76467da2ed1183f127d5f8', 'request_id': 'req-10cba0d2-d245-400c-9d60-958c369cea69'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.224681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af8188ddb76467da2ed1183f127d5f8', 'request_id': 'req-10cba0d2-d245-400c-9d60-958c369cea69'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.246678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] The token cache is being invalidate because project 0af8188ddb76467da2ed1183f127d5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:22.247161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.252501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.257507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 938/1887] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:21 2022] DELETE /identity/v3/projects/0af8188ddb76467da2ed1183f127d5f8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:22.258453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:22.260275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.296209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 950/1888] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:39:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:22.297419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:22.302385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.315813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:22.339089 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5f9f5-5887-4a2e-b13a-e14fc36b5913 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:22.355421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:22.357721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:22.358936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:22.360284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] PATH_INFO: `/v3/users/ef3177a8ead542f1800433cf7b377212` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:22.365803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cba0d2-d245-400c-9d60-958c369cea69 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:22.370750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3177a8ead542f1800433cf7b377212)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:22.373777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:22.375929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:22.377878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:22.377878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:22.377878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] PATH_INFO: `/v3/users/86167042862443eda668b2f5ef2cb015` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:22.380203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86167042862443eda668b2f5ef2cb015)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:22.384634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:22.514316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86167042862443eda668b2f5ef2cb015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.516881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3177a8ead542f1800433cf7b377212'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.525536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86167042862443eda668b2f5ef2cb015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.536353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3177a8ead542f1800433cf7b377212'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.542035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86167042862443eda668b2f5ef2cb015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.543221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86167042862443eda668b2f5ef2cb015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.553477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3177a8ead542f1800433cf7b377212'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.559133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3177a8ead542f1800433cf7b377212'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.567184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86167042862443eda668b2f5ef2cb015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.589337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 939/1889] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:22 2022] DELETE /identity/v3/users/86167042862443eda668b2f5ef2cb015 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:22.590506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:22.593351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.594154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ed0cc532-1ce1-4415-9274-be85df9b3517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3177a8ead542f1800433cf7b377212'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.608299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 951/1890] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:22 2022] DELETE /identity/v3/users/ef3177a8ead542f1800433cf7b377212 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:22.610090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:22.610556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:22.611004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:22.657721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:22.659700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.745600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c1a8e-9e85-4ffd-a674-ed81b39d0b08 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:22.766341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 952/1891] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:22.767259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:22.768673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.781499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8d807-113e-4f24-a3a9-8adae8499508 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:22.786389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:22.787262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:22.787541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:22.787807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] PATH_INFO: `/v3/projects/ee76bb8d4b804bd794696fdecc929e6d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:22.789018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee76bb8d4b804bd794696fdecc929e6d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:22.791098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:22.793142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:22.847147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:22.861503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee76bb8d4b804bd794696fdecc929e6d', 'request_id': 'req-538693f6-43ab-4240-9f6c-c449dbfdb04b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.873264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:22.874145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:22.874654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:22.875047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:22.877355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.883496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee76bb8d4b804bd794696fdecc929e6d', 'request_id': 'req-538693f6-43ab-4240-9f6c-c449dbfdb04b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.898191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:22.899572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee76bb8d4b804bd794696fdecc929e6d', 'request_id': 'req-538693f6-43ab-4240-9f6c-c449dbfdb04b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.902432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:22.924824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] The token cache is being invalidate because project ee76bb8d4b804bd794696fdecc929e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:22.925176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.925900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:22.927719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 953/1892] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:22 2022] DELETE /identity/v3/projects/ee76bb8d4b804bd794696fdecc929e6d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:22.928600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:22.933556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 940/1893] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:22.936688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:22.946499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:22.966464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4dc46-c674-4e26-b105-d4cab4c813a8 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:23.067986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:23.069254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:23.069543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:23.070204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] PATH_INFO: `/v3/users/600f6c6b83ce40849f61576b195b2b9b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:23.071617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600f6c6b83ce40849f61576b195b2b9b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:23.074306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:23.178329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600f6c6b83ce40849f61576b195b2b9b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:23.186064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538693f6-43ab-4240-9f6c-c449dbfdb04b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:23.195191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600f6c6b83ce40849f61576b195b2b9b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:23.198106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:23.198781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:23.199063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:23.199342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:23.206804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:23.209105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600f6c6b83ce40849f61576b195b2b9b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:23.209687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600f6c6b83ce40849f61576b195b2b9b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:23.227951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:23.230061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:23.233825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600f6c6b83ce40849f61576b195b2b9b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:23.247036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 941/1894] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:22 2022] DELETE /identity/v3/users/600f6c6b83ce40849f61576b195b2b9b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:23.263466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 954/1895] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:23.827148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:23.831566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:23.986997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:23.997421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034861a-a5eb-4d00-bbf9-3d2ee641c932 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:24.006479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:24.008037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:24.008795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:24.009216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] PATH_INFO: `/v3/projects/2943282a396c472b8191113b7960106a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:24.011872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2943282a396c472b8191113b7960106a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:24.019661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:24.050979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:24.052391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:24.074270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e77dc3-87f3-4792-8a1a-9453a9230b2e service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:24.080496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:24.084127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:24.084580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:24.085769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] PATH_INFO: `/v3/projects/2b4f2e7848894ed798faae6e4e521d62` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:24.088161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4f2e7848894ed798faae6e4e521d62)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:24.101246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:24.142410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2943282a396c472b8191113b7960106a', 'request_id': 'req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.161209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2943282a396c472b8191113b7960106a', 'request_id': 'req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.171117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2943282a396c472b8191113b7960106a', 'request_id': 'req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.194436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] The token cache is being invalidate because project 2943282a396c472b8191113b7960106a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:24.195862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.197699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.200387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 942/1896] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:23 2022] DELETE /identity/v3/projects/2943282a396c472b8191113b7960106a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:24.204704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:24.210786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:24.213931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4f2e7848894ed798faae6e4e521d62', 'request_id': 'req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.225881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4f2e7848894ed798faae6e4e521d62', 'request_id': 'req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.236071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4f2e7848894ed798faae6e4e521d62', 'request_id': 'req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.278798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] The token cache is being invalidate because project 2b4f2e7848894ed798faae6e4e521d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:24.279378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.280254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.282344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 955/1897] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:24 2022] DELETE /identity/v3/projects/2b4f2e7848894ed798faae6e4e521d62 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:24.286403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:24.288335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:24.318469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59901bd-ec64-4c55-84ef-e7ae0931d2e3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:24.326863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:24.430529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec03c31-753f-44e2-a1d6-a4ec6b7761f5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:24.441605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:24.442992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:24.443430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:24.443860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] PATH_INFO: `/v3/users/6efac012ba0c4da08ad6456c839c04b4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:24.448626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efac012ba0c4da08ad6456c839c04b4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:24.449541 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:24.451134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:24.451568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:24.452105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:24.452898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] PATH_INFO: `/v3/users/26e2b40f4a0846eab6ce4634ca59b3be` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:24.455709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e2b40f4a0846eab6ce4634ca59b3be)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:24.459598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:24.549082 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:24.555830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efac012ba0c4da08ad6456c839c04b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.563892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e2b40f4a0846eab6ce4634ca59b3be'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.572485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efac012ba0c4da08ad6456c839c04b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.577093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e2b40f4a0846eab6ce4634ca59b3be'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.587130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efac012ba0c4da08ad6456c839c04b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.592994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efac012ba0c4da08ad6456c839c04b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.599351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e2b40f4a0846eab6ce4634ca59b3be'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.599958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e2b40f4a0846eab6ce4634ca59b3be'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.618361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ef25d291-6ddf-4350-861c-3337c6cb9dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efac012ba0c4da08ad6456c839c04b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.622294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e2b40f4a0846eab6ce4634ca59b3be'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:24.643644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 943/1899] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:24 2022] DELETE /identity/v3/users/26e2b40f4a0846eab6ce4634ca59b3be => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:24.644001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 956/1899] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:24 2022] DELETE /identity/v3/users/6efac012ba0c4da08ad6456c839c04b4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:24.805494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:24.807126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:24.952570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4ad70-5ec2-4e36-aa00-23911d64207e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:24.957777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:24.958798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:24.959153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:24.959502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] PATH_INFO: `/v3/users/068fa4f424cd470288bb8de924373151` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:24.961463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068fa4f424cd470288bb8de924373151)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:24.963870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:25.032190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068fa4f424cd470288bb8de924373151'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.043300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068fa4f424cd470288bb8de924373151'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.059007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068fa4f424cd470288bb8de924373151'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.059979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068fa4f424cd470288bb8de924373151'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.077751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068fa4f424cd470288bb8de924373151'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.086351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 944/1900] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:24 2022] DELETE /identity/v3/users/068fa4f424cd470288bb8de924373151 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:25.091719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:25.092338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:25.092557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:25.135712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:25.137295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:25.147604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:25.149461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:25.233124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:25.278569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d2cf8-ba01-4d17-817b-7f65760be5f3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:25.280897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 957/1901] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:25.285440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:25.286716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:25.287301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:25.287715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] PATH_INFO: `/v3/projects/29e1d1a78d4b4555b273e69c61163b41` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:25.289242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e1d1a78d4b4555b273e69c61163b41)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:25.292413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:25.296756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:25.299652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:25.415135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:25.460363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e1d1a78d4b4555b273e69c61163b41', 'request_id': 'req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.484605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e1d1a78d4b4555b273e69c61163b41', 'request_id': 'req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.500010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e1d1a78d4b4555b273e69c61163b41', 'request_id': 'req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.506029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bee1c-1411-483e-9a33-0d5d56e3812a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:25.531629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] The token cache is being invalidate because project 29e1d1a78d4b4555b273e69c61163b41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:25.532144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.533058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.536288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 945/1902] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:25 2022] DELETE /identity/v3/projects/29e1d1a78d4b4555b273e69c61163b41 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:25.541153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:25.543254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:25.670344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:25.671854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:25.672191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:25.672477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:25.682830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:25.737543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f9ca3-2a5d-4e83-ba37-ae07eac49902 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:25.750605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:25.751552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:25.751839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:25.752114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] PATH_INFO: `/v3/users/6e895c9ae0274f2481aaa253d63c1be1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:25.757948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e895c9ae0274f2481aaa253d63c1be1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:25.762529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:25.894593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:25.896525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e895c9ae0274f2481aaa253d63c1be1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.897975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:25.915052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e895c9ae0274f2481aaa253d63c1be1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.928768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 958/1903] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:25.929693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:25.932095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:25.943668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e895c9ae0274f2481aaa253d63c1be1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.944479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e895c9ae0274f2481aaa253d63c1be1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.949299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:25.971573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a584818-9cfe-4811-a484-053704021725 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:25.973520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e895c9ae0274f2481aaa253d63c1be1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:25.980557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:25.981611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:25.981942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:25.982227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] PATH_INFO: `/v3/projects/f9dc28645dee4e5a92d59b3e994ed7c0` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:25.983552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 946/1904] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:25 2022] DELETE /identity/v3/users/6e895c9ae0274f2481aaa253d63c1be1 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:25.984850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:25.986808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:25.987596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9dc28645dee4e5a92d59b3e994ed7c0)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:25.990047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.065890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489ee45-ef68-4f86-aeed-9a101a144932 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.073622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9dc28645dee4e5a92d59b3e994ed7c0', 'request_id': 'req-61eb7a07-f279-459e-9ce1-c7230429efab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.095614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9dc28645dee4e5a92d59b3e994ed7c0', 'request_id': 'req-61eb7a07-f279-459e-9ce1-c7230429efab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.105433 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9dc28645dee4e5a92d59b3e994ed7c0', 'request_id': 'req-61eb7a07-f279-459e-9ce1-c7230429efab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.123679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] The token cache is being invalidate because project f9dc28645dee4e5a92d59b3e994ed7c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:26.124338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.125188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-61eb7a07-f279-459e-9ce1-c7230429efab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.132338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 959/1905] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:25 2022] DELETE /identity/v3/projects/f9dc28645dee4e5a92d59b3e994ed7c0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:26.135721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfff35-ee03-4790-a0a6-2b0a8d63bf0f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.136011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfff35-ee03-4790-a0a6-2b0a8d63bf0f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.136296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfff35-ee03-4790-a0a6-2b0a8d63bf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.152951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.153872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.154240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.154526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] PATH_INFO: `/v3/projects/d268f21bb06e42af8455dc41bcf25c97` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.156426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d268f21bb06e42af8455dc41bcf25c97)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.158218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.173789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-abcfff35-ee03-4790-a0a6-2b0a8d63bf0f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:26.177287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-abcfff35-ee03-4790-a0a6-2b0a8d63bf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:26.190482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 960/1906] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:26.199986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b67d6-3fa7-4577-aeaa-d51f95d5607a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.200296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b67d6-3fa7-4577-aeaa-d51f95d5607a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.200572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b67d6-3fa7-4577-aeaa-d51f95d5607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.238264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8e2b67d6-3fa7-4577-aeaa-d51f95d5607a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:26.245593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd268f21bb06e42af8455dc41bcf25c97', 'request_id': 'req-967be887-6b2c-462e-afa3-2f8752990a30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.256620 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd268f21bb06e42af8455dc41bcf25c97', 'request_id': 'req-967be887-6b2c-462e-afa3-2f8752990a30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.258747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8e2b67d6-3fa7-4577-aeaa-d51f95d5607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:26.266101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd268f21bb06e42af8455dc41bcf25c97', 'request_id': 'req-967be887-6b2c-462e-afa3-2f8752990a30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.280914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] The token cache is being invalidate because project d268f21bb06e42af8455dc41bcf25c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:26.281210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.283014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-967be887-6b2c-462e-afa3-2f8752990a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:26.285592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 947/1907] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:25 2022] DELETE /identity/v3/projects/d268f21bb06e42af8455dc41bcf25c97 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:26.293274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211ad3c-0708-41b6-b506-5256fb2f2c9f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.293583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211ad3c-0708-41b6-b506-5256fb2f2c9f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.293888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211ad3c-0708-41b6-b506-5256fb2f2c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.294358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:26.318952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8211ad3c-0708-41b6-b506-5256fb2f2c9f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:26.320733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8211ad3c-0708-41b6-b506-5256fb2f2c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:26.340118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:26.391603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 961/1908] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:26.392877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b67d6-3fa7-4577-aeaa-d51f95d5607a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.407234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b67d6-3fa7-4577-aeaa-d51f95d5607a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.413518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.414485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.415228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.415338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 948/1909] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:26.415653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.416569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:26.416882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.417272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bc40c-a0a1-4b65-9e3c-8205d280b793 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.417623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bc40c-a0a1-4b65-9e3c-8205d280b793 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.417979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bc40c-a0a1-4b65-9e3c-8205d280b793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.418911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.441889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 962/1910] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:26.442279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d94bc40c-a0a1-4b65-9e3c-8205d280b793 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:26.442460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.443641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d94bc40c-a0a1-4b65-9e3c-8205d280b793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:26.455063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 949/1911] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:26.457623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054a083-deb6-4aed-890d-5058f2e34136 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.459486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bc40c-a0a1-4b65-9e3c-8205d280b793 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.463392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.464279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.464666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.465192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.466160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.469028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bc40c-a0a1-4b65-9e3c-8205d280b793 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.469307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.475057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.475818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.476365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.476514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.477148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:26.477461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.479712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.494221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 950/1912] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:39:26.498294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.514850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 963/1913] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:26.515060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de0e74-efeb-4a10-87fe-974f63c1158a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.516068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.629043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a81ca5-b7e2-439a-a7b0-f41d0f08eb4e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.634056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.635533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.635854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.636139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.636845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.638999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.644847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.645537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.645825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.646126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.646838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.651533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.687979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 964/1914] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:26.691872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.706665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25084-92fb-4e09-9f5e-ad7163ec72f8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.711377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 951/1915] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:26.716078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.717045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.718528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.720602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.720602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.720602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.722930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.737794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 965/1916] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:26.743119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.754071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bd2a9-55ff-49b4-b951-cd424708d7d0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.807133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5620e-960e-4ac8-9623-9713d3f2c1f9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.813667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.814384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.814673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.814953 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.815682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.818427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.819214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.819906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.820198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.820481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.821214 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.822849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.825465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:26.839667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 966/1917] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:26.841577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.855057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7713c4-bdd4-40d2-9fe3-48069b8bf846 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.855904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 952/1918] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:26.859677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.860798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5463163-adc1-4813-b7fe-098f21bdcb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.862057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463163-adc1-4813-b7fe-098f21bdcb3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.862349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463163-adc1-4813-b7fe-098f21bdcb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.862631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463163-adc1-4813-b7fe-098f21bdcb3a admin admin] PATH_INFO: `/v3/projects/83684d39f2ed4c4da792baf2b09b7352/users/e7252a9d15f7442580bc6f60170cadfc/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.871575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25324d6-3122-430c-bec8-361149fad494 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.876497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.877184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.877913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.877913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.878637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.880266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.888358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5463163-adc1-4813-b7fe-098f21bdcb3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83684d39f2ed4c4da792baf2b09b7352, user_id=e7252a9d15f7442580bc6f60170cadfc, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.891578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5463163-adc1-4813-b7fe-098f21bdcb3a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:26.895426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 953/1919] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:26.896391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:26.905587 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:26.910751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad5e66-fba1-4c44-9e88-cdd3486c0555 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:26.920907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 967/1920] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:26 2022] PUT /identity/v3/projects/83684d39f2ed4c4da792baf2b09b7352/users/e7252a9d15f7442580bc6f60170cadfc/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:26.932542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6427-2f0e-4d03-b3b1-8f7a69949e61 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.932844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6427-2f0e-4d03-b3b1-8f7a69949e61 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.933123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f6427-2f0e-4d03-b3b1-8f7a69949e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.978950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-943f6427-2f0e-4d03-b3b1-8f7a69949e61 None None] MFA Rules not processed for user `e7252a9d15f7442580bc6f60170cadfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:26.980304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-943f6427-2f0e-4d03-b3b1-8f7a69949e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:26.988738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:26.989520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:26.989914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:26.990272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:26.991145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:26.993152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.005879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 954/1921] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:27.010129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.022715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56859f-14e9-4872-9fe7-c607555e4156 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.028994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effe2a7-ca28-4c1d-8d0b-1c45531e9547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.030135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe2a7-ca28-4c1d-8d0b-1c45531e9547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.030629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe2a7-ca28-4c1d-8d0b-1c45531e9547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.031058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effe2a7-ca28-4c1d-8d0b-1c45531e9547 admin admin] PATH_INFO: `/v3/projects/af7df8c002c94557ae02dab16b6b5ba3/users/673271791e5c4b058af849ef441e4dab/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.053742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effe2a7-ca28-4c1d-8d0b-1c45531e9547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7df8c002c94557ae02dab16b6b5ba3, user_id=673271791e5c4b058af849ef441e4dab, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.058015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effe2a7-ca28-4c1d-8d0b-1c45531e9547 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.088017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 955/1922] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:27 2022] PUT /identity/v3/projects/af7df8c002c94557ae02dab16b6b5ba3/users/673271791e5c4b058af849ef441e4dab/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:27.090208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.090839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.090958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.121257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] MFA Rules not processed for user `673271791e5c4b058af849ef441e4dab`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:27.122669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.143686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 968/1923] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:27.144634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f6427-2f0e-4d03-b3b1-8f7a69949e61 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.169052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f6427-2f0e-4d03-b3b1-8f7a69949e61 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.256319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 956/1924] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:27.258132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.262098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.294636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.296045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.296438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.297363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.298208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.301614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.342738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 969/1925] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:27 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:27.343678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.346374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.348062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81330c80-feca-4e04-9644-5dbc04f7ba2f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.358179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:27.397276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:27.409390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.410356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.410641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.410919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] PATH_INFO: `/v3/projects/a84751c2bbd747519f1f4807b30738d0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.412144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84751c2bbd747519f1f4807b30738d0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.413463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8792f-e6c6-4aef-ab34-53351ba3e959 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.414165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.418847 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.419709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.419990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.420261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] PATH_INFO: `/v3/projects/994b68ab31514facb77713ed4c74fe97` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.421452 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] RBAC: Authorizing `identity:delete_project(project_id=994b68ab31514facb77713ed4c74fe97)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.423358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.463516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84751c2bbd747519f1f4807b30738d0', 'request_id': 'req-922a09cc-ad9e-420b-985f-a3aace727399'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.471799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84751c2bbd747519f1f4807b30738d0', 'request_id': 'req-922a09cc-ad9e-420b-985f-a3aace727399'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.472303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '994b68ab31514facb77713ed4c74fe97', 'request_id': 'req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.478531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84751c2bbd747519f1f4807b30738d0', 'request_id': 'req-922a09cc-ad9e-420b-985f-a3aace727399'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.479658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '994b68ab31514facb77713ed4c74fe97', 'request_id': 'req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.485259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '994b68ab31514facb77713ed4c74fe97', 'request_id': 'req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.490212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] The token cache is being invalidate because project a84751c2bbd747519f1f4807b30738d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:27.490506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.491164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.492753 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 957/1926] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:27 2022] DELETE /identity/v3/projects/a84751c2bbd747519f1f4807b30738d0 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:27.493626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.497504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] The token cache is being invalidate because project 994b68ab31514facb77713ed4c74fe97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:27.497688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.497788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.498509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.503279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 970/1927] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:27 2022] DELETE /identity/v3/projects/994b68ab31514facb77713ed4c74fe97 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:27.503279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.504048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.560846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a09cc-ad9e-420b-985f-a3aace727399 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.578935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029d3dc-aa4f-4e69-b2fe-dd1b0e5c848f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.583835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766955e3-17c9-4abc-bb0c-2c2345ac4d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.584508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766955e3-17c9-4abc-bb0c-2c2345ac4d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.584789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766955e3-17c9-4abc-bb0c-2c2345ac4d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.585066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766955e3-17c9-4abc-bb0c-2c2345ac4d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.585756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766955e3-17c9-4abc-bb0c-2c2345ac4d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.588270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766955e3-17c9-4abc-bb0c-2c2345ac4d3a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.610305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 971/1928] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:27 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:27.611599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.611887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.612162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.614613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.615274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.615558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.615845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.617627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.631844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:27.640912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:27.642483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.652714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 972/1929] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:27.653828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.655236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.670870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1181b7-5eb6-4f43-9a9b-91b798f776ff None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.676734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.678195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.678479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.678750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] PATH_INFO: `/v3/users/cf48104be3054ee89a9ee2e0afc3314f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.680318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf48104be3054ee89a9ee2e0afc3314f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.682389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.708450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.711765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.735189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 958/1930] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:27.736910 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.753395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf48104be3054ee89a9ee2e0afc3314f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.764887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf48104be3054ee89a9ee2e0afc3314f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.768254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:27.776047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf48104be3054ee89a9ee2e0afc3314f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.776979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf48104be3054ee89a9ee2e0afc3314f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.778214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8380b6c-49de-47e2-b3f4-943ed7b76913 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.796825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.797528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.797815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.798118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.798867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.807617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c778e84-20e9-42f6-bca5-68346c0a37d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf48104be3054ee89a9ee2e0afc3314f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:27.810376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.817308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 973/1931] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:27 2022] DELETE /identity/v3/users/cf48104be3054ee89a9ee2e0afc3314f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:27.817308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8dcc1-9b85-482e-be2c-8060921a0edd None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.817308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8dcc1-9b85-482e-be2c-8060921a0edd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.817308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8dcc1-9b85-482e-be2c-8060921a0edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.839147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dad8dcc1-9b85-482e-be2c-8060921a0edd None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:27.840527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dad8dcc1-9b85-482e-be2c-8060921a0edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.852967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 959/1932] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:27.854578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.863158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:27.905125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f4575-5aef-4473-a396-0f7a08579d8d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.913959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.914936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.915153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.915457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] PATH_INFO: `/v3/projects/8fd8cdc3850a4453b10a65d434e0e814/users/ede2d553ca9b4cd8b21e741a90319917/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.916338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 974/1933] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:27.917161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8dcc1-9b85-482e-be2c-8060921a0edd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.927775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8dcc1-9b85-482e-be2c-8060921a0edd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.933986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:27.934530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:27.934813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:27.935270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:27.936365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:27.936687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.937443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd8cdc3850a4453b10a65d434e0e814, user_id=ede2d553ca9b4cd8b21e741a90319917, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:27.938417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.940267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:27.953467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 975/1934] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:27.958394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:27.968383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 960/1935] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:27 2022] PUT /identity/v3/projects/8fd8cdc3850a4453b10a65d434e0e814/users/ede2d553ca9b4cd8b21e741a90319917/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:27.971540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb776f-cbbc-4411-ac64-b603b6941f67 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:27.972826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.043967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.044922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.045297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.045636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.046485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.049355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.058146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81818c55-d495-44b0-9efa-2382a4b5fc67 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.063138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.064034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.064322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.064602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.065369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.067053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.082601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 961/1936] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:28.083555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.092827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 976/1937] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:27 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:28.094586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.096793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7485474-59ec-44b8-b46a-d1c0ce772628 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.164703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:28.177075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c4db5-f486-445c-b81a-d6f8c0dab385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.178173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c4db5-f486-445c-b81a-d6f8c0dab385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.178456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c4db5-f486-445c-b81a-d6f8c0dab385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.178845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c4db5-f486-445c-b81a-d6f8c0dab385 admin admin] PATH_INFO: `/v3/projects/8fd8cdc3850a4453b10a65d434e0e814/users/ede2d553ca9b4cd8b21e741a90319917/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.181702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c4db5-f486-445c-b81a-d6f8c0dab385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd8cdc3850a4453b10a65d434e0e814, user_id=ede2d553ca9b4cd8b21e741a90319917, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.184828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c4db5-f486-445c-b81a-d6f8c0dab385 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.193466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270809a-0c3d-45be-a733-9750446f1127 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.211286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 962/1938] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:28 2022] PUT /identity/v3/projects/8fd8cdc3850a4453b10a65d434e0e814/users/ede2d553ca9b4cd8b21e741a90319917/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:28.216956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.217272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.219658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.317056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:28.336216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.337627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.338208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.338747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.339710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.348176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] MFA Rules not processed for user `ede2d553ca9b4cd8b21e741a90319917`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:28.350015 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.350760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:28.404675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 977/1939] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:28 2022] POST /identity/v3/users => generated 282 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:28.410954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.437655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b7f39-8342-464f-81cd-12fa5f5008a3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.444142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.444959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.445308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.445653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.446509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.448249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.460537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 978/1940] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:28.464406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.475202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2a1d9-812f-4c77-beff-a88b5f586bc8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.481438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.482468 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.482804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.483175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.484069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.485750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.502953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 979/1941] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:28.503139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 963/1942] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:28.504526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.515165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3607f7-10d0-4dab-a138-fd39ca185e19 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.516489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.519414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:28.521863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cb284-c36c-409f-abfe-ac20a56d40a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.522075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cb284-c36c-409f-abfe-ac20a56d40a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.522483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cb284-c36c-409f-abfe-ac20a56d40a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.522833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cb284-c36c-409f-abfe-ac20a56d40a6 admin admin] PATH_INFO: `/v3/projects/4053557239484ae5b823850d9b12f5b2/users/66dc033b9ed4472f9e3948a1bccc54f8/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.542552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cb284-c36c-409f-abfe-ac20a56d40a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4053557239484ae5b823850d9b12f5b2, user_id=66dc033b9ed4472f9e3948a1bccc54f8, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.545696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cb284-c36c-409f-abfe-ac20a56d40a6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.569936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 980/1943] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:28 2022] PUT /identity/v3/projects/4053557239484ae5b823850d9b12f5b2/users/66dc033b9ed4472f9e3948a1bccc54f8/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:28.580332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.580654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.580893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.593699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa705d-12a3-4da0-a69f-d5972c7e5b58 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.609855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] MFA Rules not processed for user `66dc033b9ed4472f9e3948a1bccc54f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:28.614931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:28.661674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.662605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.663144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.663426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.695045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 981/1944] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:28 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:28.695045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.695045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:28.725344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:28.725995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.727810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.733878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:28.739399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 964/1945] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:28.740521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.768019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb383c75-af61-4a8d-81ee-c17f3a49e9c2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.772956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.774094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.776676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.776988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] PATH_INFO: `/v3/projects/4fc651f0622a4a38b0e8aadb76b259be` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.785158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc651f0622a4a38b0e8aadb76b259be)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.787087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.815080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fd094-a1e0-49c1-bb7e-fdf21938b78b service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.820533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.821376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.821654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.821942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.822646 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:28.824810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:28.843659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc651f0622a4a38b0e8aadb76b259be', 'request_id': 'req-5bc001ef-a8f0-4f04-924f-b6107389ed36'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:28.855656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc651f0622a4a38b0e8aadb76b259be', 'request_id': 'req-5bc001ef-a8f0-4f04-924f-b6107389ed36'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:28.861566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 965/1946] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:28 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:28.861566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.864425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc651f0622a4a38b0e8aadb76b259be', 'request_id': 'req-5bc001ef-a8f0-4f04-924f-b6107389ed36'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:28.876575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] The token cache is being invalidate because project 4fc651f0622a4a38b0e8aadb76b259be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:28.876865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:28.877594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:28.879222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 982/1947] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:28 2022] DELETE /identity/v3/projects/4fc651f0622a4a38b0e8aadb76b259be => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:28.880050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:28.881368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:28.955299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acb620-b63d-4d02-ad1b-c4ce5e83e7d4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:28.964474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e73ed9-bec1-41c1-825a-2086dcf572fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:28.965283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73ed9-bec1-41c1-825a-2086dcf572fe service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:28.965642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73ed9-bec1-41c1-825a-2086dcf572fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:28.966068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e73ed9-bec1-41c1-825a-2086dcf572fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:28.970922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-71e73ed9-bec1-41c1-825a-2086dcf572fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:28.986291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc001ef-a8f0-4f04-924f-b6107389ed36 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.047771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e73ed9-bec1-41c1-825a-2086dcf572fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.049412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e73ed9-bec1-41c1-825a-2086dcf572fe service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.059138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 966/1948] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:29.060143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.061193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.061681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.062172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.063067 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.066306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.069135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b8e0-3885-48bb-9180-f3b7521ed852 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.069449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b8e0-3885-48bb-9180-f3b7521ed852 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.069706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018b8e0-3885-48bb-9180-f3b7521ed852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.088589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:29.106706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1018b8e0-3885-48bb-9180-f3b7521ed852 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:29.108821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1018b8e0-3885-48bb-9180-f3b7521ed852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:29.109584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 983/1949] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:28 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:29.110662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.117924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 967/1950] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:29.122741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439084e9-b5f3-4c95-a8f9-943488e3fa89 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.125670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cac551-8087-4043-82a8-e2fb2498d841 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.126447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cac551-8087-4043-82a8-e2fb2498d841 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.126734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cac551-8087-4043-82a8-e2fb2498d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.128361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.129035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.129317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.129738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.130362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.132045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.144675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 984/1951] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:29.149320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.152790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d9cac551-8087-4043-82a8-e2fb2498d841 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:29.154545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d9cac551-8087-4043-82a8-e2fb2498d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:29.165320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38c383-d3c5-412b-a59f-cdf83259eadc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.171561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.172302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.172762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.173146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] PATH_INFO: `/v3/projects/a0cfe7a3b37742e38d8b7a0038a436e9/users/6f4993c486604dd882141fddee0674ec/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.178408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 968/1952] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:29.182346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cac551-8087-4043-82a8-e2fb2498d841 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.191502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0cfe7a3b37742e38d8b7a0038a436e9, user_id=6f4993c486604dd882141fddee0674ec, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.194076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cac551-8087-4043-82a8-e2fb2498d841 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.194641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.203257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.203945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.204251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.204480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.205325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:29.205706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.207631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.216105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 985/1953] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:29 2022] PUT /identity/v3/projects/a0cfe7a3b37742e38d8b7a0038a436e9/users/6f4993c486604dd882141fddee0674ec/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:29.217020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.235918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 969/1954] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:39:29.236320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.255677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde5f0-a69b-4912-a47c-725d5f49affe admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.286618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:29.294586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc1c48-29ab-466d-971e-ec770240bf29 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.300885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.301748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.302734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.302961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.303883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.305771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.325093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 986/1955] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:29.326974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.346114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c824e3-5ef1-4792-8a20-d7e7e5e635a5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.363633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fd797-9bb2-4eba-aa0d-0b94704a8b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.365048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fd797-9bb2-4eba-aa0d-0b94704a8b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.365320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fd797-9bb2-4eba-aa0d-0b94704a8b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.365884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fd797-9bb2-4eba-aa0d-0b94704a8b32 admin admin] PATH_INFO: `/v3/projects/a0cfe7a3b37742e38d8b7a0038a436e9/users/6f4993c486604dd882141fddee0674ec/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.373196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fd797-9bb2-4eba-aa0d-0b94704a8b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0cfe7a3b37742e38d8b7a0038a436e9, user_id=6f4993c486604dd882141fddee0674ec, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.377957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fd797-9bb2-4eba-aa0d-0b94704a8b32 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.407544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.408709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 987/1956] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:29 2022] PUT /identity/v3/projects/a0cfe7a3b37742e38d8b7a0038a436e9/users/6f4993c486604dd882141fddee0674ec/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:29.418036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.418036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.418036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.418036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.418036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.420180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.420589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.420930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.496800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 970/1957] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:29 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:29.500680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.513853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] MFA Rules not processed for user `6f4993c486604dd882141fddee0674ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:29.515661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:29.668550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bbb3e-7cf6-4983-9f1b-b3d1b4a35dbf admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.684330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.685693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.686029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.686309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.687024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.692619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.710485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 988/1958] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:29.717524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.720598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:29.737152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 971/1959] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:29 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:29.738234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.741394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:29.758769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f24d74-35f6-4f0c-82ad-383ee9896f86 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.767388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.768273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.768563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.768836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] PATH_INFO: `/v3/users/e7252a9d15f7442580bc6f60170cadfc` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.772548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7252a9d15f7442580bc6f60170cadfc)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.776530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.800750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=3,keystone:SELECT=94,keystone:DELETE=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:29.840199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d34ad-3ba0-4c52-99b5-36b2ba74ec35 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.847220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.848287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.848458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7252a9d15f7442580bc6f60170cadfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:29.848800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.849214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.862254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7252a9d15f7442580bc6f60170cadfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:29.866445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.868460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.872615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7252a9d15f7442580bc6f60170cadfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:29.874073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7252a9d15f7442580bc6f60170cadfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:29.890023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 972/1960] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:29 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:29.890894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.894121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abe251e9-b339-4266-aa45-16b8e0a8879d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7252a9d15f7442580bc6f60170cadfc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:29.906675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6ef98-e992-48ed-8e56-0543c8cc9353 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.917167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 989/1961] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:29 2022] DELETE /identity/v3/users/e7252a9d15f7442580bc6f60170cadfc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:29.918343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:29.919032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.919318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.919596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.920344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:29.921751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:29.922112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:29.922217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:29.923900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:29.937354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 973/1962] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:29.943143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:29.945114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:29.949118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:29.950357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:29.961461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7aa6-3fcf-4fa0-bb1e-245c7399e665 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:29.986423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:30.047775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 990/1963] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:30.048523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.049831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:30.050702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.051443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.051732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.052456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.052760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.054430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.078212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 974/1964] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:30.079242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.081422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:30.146828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61ac75-4cdd-4686-93d3-3fa1d1694765 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.151411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aa721-2c90-48c4-9edb-86f1fd80651e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.152080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa721-2c90-48c4-9edb-86f1fd80651e service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.152364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa721-2c90-48c4-9edb-86f1fd80651e service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.152641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aa721-2c90-48c4-9edb-86f1fd80651e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.154531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f54aa721-2c90-48c4-9edb-86f1fd80651e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:30.161595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07bfb5-25b8-40f9-affc-0ca3e4192b3a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.166768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.167675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.167973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.168245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.178471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.179906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.196251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 975/1965] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:30.197440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.208501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ced94b-e466-4ceb-a3c1-a3287e6f769b service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.215920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.216928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.217985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.217985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] PATH_INFO: `/v3/projects/932c2e011cc249faa4b20aaed4f346fc/users/35c62b58564442e8a9c523e763ddd26f/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.222517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:30.228755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa721-2c90-48c4-9edb-86f1fd80651e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.231952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aa721-2c90-48c4-9edb-86f1fd80651e service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.240437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932c2e011cc249faa4b20aaed4f346fc, user_id=35c62b58564442e8a9c523e763ddd26f, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.244102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.250399 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 991/1966] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:30.263179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 976/1967] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:30 2022] PUT /identity/v3/projects/932c2e011cc249faa4b20aaed4f346fc/users/35c62b58564442e8a9c523e763ddd26f/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:30.274831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556c5e4-8999-404c-b22a-133b66b73ed8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.275184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556c5e4-8999-404c-b22a-133b66b73ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.275471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556c5e4-8999-404c-b22a-133b66b73ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.311374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9556c5e4-8999-404c-b22a-133b66b73ed8 None None] MFA Rules not processed for user `35c62b58564442e8a9c523e763ddd26f`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:30.312652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9556c5e4-8999-404c-b22a-133b66b73ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:30.408407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 992/1968] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:30.412793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.491061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ecf00-94e4-4c90-b1b2-f3b316e706c4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.503427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.504316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.504755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.505051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.505970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.508811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.539774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 977/1969] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:30 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:30.543688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556c5e4-8999-404c-b22a-133b66b73ed8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.557492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556c5e4-8999-404c-b22a-133b66b73ed8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.625766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.627743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:30.669962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.670736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.671022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.671600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.672372 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:30.674873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.677685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.720706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 993/1970] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:30 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:30.724474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.747073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca146e-128e-4e17-9399-3d19c3bfe371 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.751885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.752903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.753289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.753808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] PATH_INFO: `/v3/projects/83684d39f2ed4c4da792baf2b09b7352` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.755186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83684d39f2ed4c4da792baf2b09b7352)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.756011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcabb5-1326-43fb-8827-0585fecfdeb2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.757332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.763615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.764655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.765080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.765494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.766901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.769636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.789325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 994/1971] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:30.790540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.801715 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fe11b-2531-4940-ae33-60693869118d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.807084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.808136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.808334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.808987 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] PATH_INFO: `/v3/projects/cced49c028d242c2b4bea773f01e3581/users/c5e8e379c4dd4aaab569f3aa7d075787/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.846644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cced49c028d242c2b4bea773f01e3581, user_id=c5e8e379c4dd4aaab569f3aa7d075787, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.848937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.856480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83684d39f2ed4c4da792baf2b09b7352', 'request_id': 'req-61facf56-df35-4316-b2e6-ef949e21d902'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:30.864948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83684d39f2ed4c4da792baf2b09b7352', 'request_id': 'req-61facf56-df35-4316-b2e6-ef949e21d902'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:30.869119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 995/1972] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:30 2022] PUT /identity/v3/projects/cced49c028d242c2b4bea773f01e3581/users/c5e8e379c4dd4aaab569f3aa7d075787/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:30.870424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.874759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83684d39f2ed4c4da792baf2b09b7352', 'request_id': 'req-61facf56-df35-4316-b2e6-ef949e21d902'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:30.887046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] The token cache is being invalidate because project 83684d39f2ed4c4da792baf2b09b7352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:30.887340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:30.888059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:30.889609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 978/1973] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:30 2022] DELETE /identity/v3/projects/83684d39f2ed4c4da792baf2b09b7352 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:30.893976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.895593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:30.926945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:30.971931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa30b0c-1cf4-4037-9ecd-6bd8a800023e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:30.977703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:30.978610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:30.978940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:30.979223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:30.979969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:30.981619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:30.997334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 996/1974] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:30.997607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:30.999659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:31.018432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4005e-2b6c-4833-ab6f-bb611b9d4ed6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:31.031695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:31.032222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61facf56-df35-4316-b2e6-ef949e21d902 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:31.032653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:31.033048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:31.033509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] PATH_INFO: `/v3/users/673271791e5c4b058af849ef441e4dab` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:31.036217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=673271791e5c4b058af849ef441e4dab)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:31.039024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:31.046017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:31.046944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:31.047230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:31.047504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] PATH_INFO: `/v3/users/ede2d553ca9b4cd8b21e741a90319917` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:31.057059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede2d553ca9b4cd8b21e741a90319917)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:31.059561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:31.126351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede2d553ca9b4cd8b21e741a90319917'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.131110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '673271791e5c4b058af849ef441e4dab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.132807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:31.137431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede2d553ca9b4cd8b21e741a90319917'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.151585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '673271791e5c4b058af849ef441e4dab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.161797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede2d553ca9b4cd8b21e741a90319917'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.163172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '673271791e5c4b058af849ef441e4dab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.163672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede2d553ca9b4cd8b21e741a90319917'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.164041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '673271791e5c4b058af849ef441e4dab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.192050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '673271791e5c4b058af849ef441e4dab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.199947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 997/1975] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:30 2022] DELETE /identity/v3/users/673271791e5c4b058af849ef441e4dab => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:31.200743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:31.203734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:31.206448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d42e06ba-0af5-4988-b773-8bc165e54b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede2d553ca9b4cd8b21e741a90319917'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:31.222565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 979/1976] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:30 2022] DELETE /identity/v3/users/ede2d553ca9b4cd8b21e741a90319917 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:31.229889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:31.230187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:31.230470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:31.272192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:31.273258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:31.321310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe10db-4edd-4a82-92f3-d9f87540d993 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:31.420342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 980/1977] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:31.436974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:31.437491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:31.438970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:31.467364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f5edb-ce93-419a-9685-a2c144ef8ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:31.469290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5edb-ce93-419a-9685-a2c144ef8ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:31.469644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5edb-ce93-419a-9685-a2c144ef8ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:31.470035 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f5edb-ce93-419a-9685-a2c144ef8ff1 admin admin] PATH_INFO: `/v3/projects/cced49c028d242c2b4bea773f01e3581/users/c5e8e379c4dd4aaab569f3aa7d075787/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:31.473435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f5edb-ce93-419a-9685-a2c144ef8ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cced49c028d242c2b4bea773f01e3581, user_id=c5e8e379c4dd4aaab569f3aa7d075787, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:31.478393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f5edb-ce93-419a-9685-a2c144ef8ff1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:31.502769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 998/1978] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:31 2022] PUT /identity/v3/projects/cced49c028d242c2b4bea773f01e3581/users/c5e8e379c4dd4aaab569f3aa7d075787/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:31.508434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:31.511437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:31.512338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:31.530306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322150c2-7707-47ef-ac4a-0986b1069b21 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:31.577515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] MFA Rules not processed for user `c5e8e379c4dd4aaab569f3aa7d075787`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:31.579218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:31.595227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:31.595919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:31.596202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:31.596478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:31.644757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:31.670139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:31.671611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:31.690135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 981/1979] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:31.760612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 999/1980] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:31.819543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:31.821148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:31.922987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8d7dc-bde7-45e1-ae52-52e3f06de3d7 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:31.927394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b07f7-b278-40ab-b052-7a641bc18bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:31.928115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b07f7-b278-40ab-b052-7a641bc18bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:31.928427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b07f7-b278-40ab-b052-7a641bc18bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:31.928666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b07f7-b278-40ab-b052-7a641bc18bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:31.931487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1f4b07f7-b278-40ab-b052-7a641bc18bb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:31.993229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:31.994810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.012008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6f512-7253-4cd5-a36b-3d81e40b2a9c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.014442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b07f7-b278-40ab-b052-7a641bc18bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.016418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b07f7-b278-40ab-b052-7a641bc18bb9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.017293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.018190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.018465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.018745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] PATH_INFO: `/v3/projects/8fd8cdc3850a4453b10a65d434e0e814` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.020058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd8cdc3850a4453b10a65d434e0e814)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.021970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.031669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 982/1981] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:32.126226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd8cdc3850a4453b10a65d434e0e814', 'request_id': 'req-a0cb854b-6909-4493-beff-dfe3f586bec4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.139640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd8cdc3850a4453b10a65d434e0e814', 'request_id': 'req-a0cb854b-6909-4493-beff-dfe3f586bec4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.147359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd8cdc3850a4453b10a65d434e0e814', 'request_id': 'req-a0cb854b-6909-4493-beff-dfe3f586bec4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.161423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] The token cache is being invalidate because project 8fd8cdc3850a4453b10a65d434e0e814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:32.161783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.162790 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.164812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1000/1982] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:31 2022] DELETE /identity/v3/projects/8fd8cdc3850a4453b10a65d434e0e814 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:32.169496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730569-f09a-47ac-9454-67da5b793590 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.170001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730569-f09a-47ac-9454-67da5b793590 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.170476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730569-f09a-47ac-9454-67da5b793590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.193530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-64730569-f09a-47ac-9454-67da5b793590 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:32.194907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-64730569-f09a-47ac-9454-67da5b793590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.276956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.279081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.284205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 983/1983] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:32.291983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca46024-0cad-4f73-99c1-9e9cb5b9fba4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.292279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca46024-0cad-4f73-99c1-9e9cb5b9fba4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.292567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca46024-0cad-4f73-99c1-9e9cb5b9fba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.312538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb854b-6909-4493-beff-dfe3f586bec4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.319301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.319441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aca46024-0cad-4f73-99c1-9e9cb5b9fba4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:32.320334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.320686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.320784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aca46024-0cad-4f73-99c1-9e9cb5b9fba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.321057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] PATH_INFO: `/v3/projects/af7df8c002c94557ae02dab16b6b5ba3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.322923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7df8c002c94557ae02dab16b6b5ba3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.325462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.333489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 984/1984] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:32.339272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca46024-0cad-4f73-99c1-9e9cb5b9fba4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.349720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca46024-0cad-4f73-99c1-9e9cb5b9fba4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.357971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.358675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.358964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.359416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.360239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:32.360545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.363970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.378946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 985/1985] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:39:32.383919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.391012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7df8c002c94557ae02dab16b6b5ba3', 'request_id': 'req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.393331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:32.395207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe3221-c669-49ca-a002-6dda8ec173f4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.401746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7df8c002c94557ae02dab16b6b5ba3', 'request_id': 'req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.406902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:32.411045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7df8c002c94557ae02dab16b6b5ba3', 'request_id': 'req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.426750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] The token cache is being invalidate because project af7df8c002c94557ae02dab16b6b5ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:32.427044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.428188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-258b6457-d7c8-4dec-a4b5-fe4bf4d2fe53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:32.429486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1001/1986] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:32 2022] DELETE /identity/v3/projects/af7df8c002c94557ae02dab16b6b5ba3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:32.436123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd13c2-ccfc-476a-b938-0311a7005501 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.436123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd13c2-ccfc-476a-b938-0311a7005501 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.436281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd13c2-ccfc-476a-b938-0311a7005501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.458220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-48dd13c2-ccfc-476a-b938-0311a7005501 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:32.459496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-48dd13c2-ccfc-476a-b938-0311a7005501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.478748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.479396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.479743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.480219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.481025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.484418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.517689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 986/1987] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:32.523166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.525204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.561991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1002/1988] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:32.569745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d805-b3f4-40f4-bf18-87d021fa0241 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.570355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d805-b3f4-40f4-bf18-87d021fa0241 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.570759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6d805-b3f4-40f4-bf18-87d021fa0241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.606430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7fb6d805-b3f4-40f4-bf18-87d021fa0241 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:32.607670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7fb6d805-b3f4-40f4-bf18-87d021fa0241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.646597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81533a99-e421-48b0-b5d7-a5e80bc8ab01 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.652112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.652887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.653247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.653598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.654616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.657360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.700346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 987/1989] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:32.704752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.712699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1003/1990] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:32.714164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d805-b3f4-40f4-bf18-87d021fa0241 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.716781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11eba0-0851-4e52-84a8-1530ac98f378 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.722136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.723237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.723622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.724001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.724851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.726851 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.727419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6d805-b3f4-40f4-bf18-87d021fa0241 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.736169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.737318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.737674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.738258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.739460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:32.740090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.740644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 988/1991] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:32.745610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.746306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.761957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1004/1992] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:32.762177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0ac0a-f22e-4cf4-b109-9660e63459ce admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.763162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.767264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.768716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.769238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.769617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.770520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.772983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.776441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d35ea4-a6ca-4592-bab1-b845c8f0e0d6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.783702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.785042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.785355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.785636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.786637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.787608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 989/1993] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:32.791617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.792248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.808368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0a61-7c31-494b-bea0-13b62d838128 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.820292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74a0e5-bfa5-45b9-af7c-10d0444984cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:32.821265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74a0e5-bfa5-45b9-af7c-10d0444984cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.821585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74a0e5-bfa5-45b9-af7c-10d0444984cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.821883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74a0e5-bfa5-45b9-af7c-10d0444984cf admin admin] PATH_INFO: `/v3/projects/a064b94b61944cec9faaad2648d247ff/users/6c60de8fd0c64a44bb71c6e98e7ede9a/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.839074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74a0e5-bfa5-45b9-af7c-10d0444984cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a064b94b61944cec9faaad2648d247ff, user_id=6c60de8fd0c64a44bb71c6e98e7ede9a, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:32.841983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74a0e5-bfa5-45b9-af7c-10d0444984cf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:32.850016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1005/1994] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/projects => generated 365 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:32.851093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:32.862425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 990/1995] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:32 2022] PUT /identity/v3/projects/a064b94b61944cec9faaad2648d247ff/users/6c60de8fd0c64a44bb71c6e98e7ede9a/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:32.867354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:32.867660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:32.867943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:32.898426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:32.901697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:32.916150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:32.940380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98eca2-9dd5-4b26-a60f-8a9648bdd5db admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:32.945390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:32.994997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 991/1996] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:33.005307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:33.007472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:33.029584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:33.030534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.030896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.031267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.032525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:33.034730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:33.070559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1006/1997] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:32 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:33.071195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:33.091365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb26a1-9c1b-49e3-a553-99a135e45a36 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:33.096796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:33.097492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.097895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.098395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.100318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:33.102322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:33.104907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65deead7-8cc5-4f28-a2d4-abaf42c344ba None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:33.109429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:33.110363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.110679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.111070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.117801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1007/1998] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:33.121531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:33.131209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:33.132017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958afd7-230f-4ab9-93cb-abf6e81077d0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:33.133472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:33.137300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea364ce4-6db3-4bc3-acac-e43f0b405c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:33.138320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea364ce4-6db3-4bc3-acac-e43f0b405c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.138632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea364ce4-6db3-4bc3-acac-e43f0b405c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.139013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea364ce4-6db3-4bc3-acac-e43f0b405c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.139859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea364ce4-6db3-4bc3-acac-e43f0b405c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:33.141616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea364ce4-6db3-4bc3-acac-e43f0b405c53 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:33.145714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 992/1999] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:33.157485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1008/2000] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:33.166548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:33.187127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac079d-3228-4005-8a3b-96dd27549a76 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:33.193397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:33.194342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.194631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.194966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] PATH_INFO: `/v3/projects/9a58e7ee487b438f8641edce5eea3d34/users/3f773aab53304103a72b27226cc1b729/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.215532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a58e7ee487b438f8641edce5eea3d34, user_id=3f773aab53304103a72b27226cc1b729, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:33.218795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:33.241941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 993/2001] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:33 2022] PUT /identity/v3/projects/9a58e7ee487b438f8641edce5eea3d34/users/3f773aab53304103a72b27226cc1b729/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:33.248902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.249134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.249406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.299525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] MFA Rules not processed for user `3f773aab53304103a72b27226cc1b729`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:33.300903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:33.398852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1009/2002] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:33.413890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:33.418014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:33.440648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:33.443083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:33.501024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06feca-6e80-4f02-87dd-3402315661a0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:33.505356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf5d7c-5dbe-4cfb-b795-16d1111f976c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:33.506218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf5d7c-5dbe-4cfb-b795-16d1111f976c service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.506526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf5d7c-5dbe-4cfb-b795-16d1111f976c service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.506787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf5d7c-5dbe-4cfb-b795-16d1111f976c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.511230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:33.518050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf5d7c-5dbe-4cfb-b795-16d1111f976c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:33.519531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf5d7c-5dbe-4cfb-b795-16d1111f976c service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:33.531415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 994/2003] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:33.537874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9e4e4-938a-4ca8-99b6-e30b9e5098e9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:33.542565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:33.543572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.543891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.544164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] PATH_INFO: `/v3/users/66dc033b9ed4472f9e3948a1bccc54f8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.545392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=66dc033b9ed4472f9e3948a1bccc54f8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:33.547302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:33.631952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66dc033b9ed4472f9e3948a1bccc54f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:33.648445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66dc033b9ed4472f9e3948a1bccc54f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:33.664084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66dc033b9ed4472f9e3948a1bccc54f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:33.665509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66dc033b9ed4472f9e3948a1bccc54f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:33.693036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66dc033b9ed4472f9e3948a1bccc54f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:33.707268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1010/2004] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:33 2022] DELETE /identity/v3/users/66dc033b9ed4472f9e3948a1bccc54f8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:33.712962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc4229-7115-49d5-b13c-b7cf3242b953 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:33.713403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc4229-7115-49d5-b13c-b7cf3242b953 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:33.713685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc4229-7115-49d5-b13c-b7cf3242b953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:33.746736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c3cc4229-7115-49d5-b13c-b7cf3242b953 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:33.749987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c3cc4229-7115-49d5-b13c-b7cf3242b953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:33.865063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 995/2005] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:33.916428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:33.918637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:34.030453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f528a7-603d-4aac-9f8d-84da3f5080ed admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:34.044481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:34.045623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.046094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.046537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.063632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:34.065664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:34.087551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1011/2006] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:34.345353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.345827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.346195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.354804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:34.388349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] MFA Rules not processed for user `6c60de8fd0c64a44bb71c6e98e7ede9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:34.390089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:34.496482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 996/2007] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:34.501328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:34.502784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:34.525192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60448-72c0-4c13-b3cf-4d58c324de4d service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:34.534434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:34.535671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.536390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.536887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.537833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:34.541148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:34.574847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:34.577679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:34.593520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1012/2008] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:34 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:34.595962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:34.606886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda05096-56ab-40a5-8a8f-6ef6205a6a3b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:34.680021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:34.697659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc089b01-d4c4-41bb-b9ce-d404888555cb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:34.701059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:34.702657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.703858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.704363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] PATH_INFO: `/v3/projects/4053557239484ae5b823850d9b12f5b2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.706220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4053557239484ae5b823850d9b12f5b2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:34.708250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:34.708537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.708993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.709275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.710181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:34.710539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:34.713122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:34.745883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1013/2009] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:34 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:34.749863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:34.763519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25ab1b-9052-4848-83ad-2c0003c8fb3f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:34.783122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4053557239484ae5b823850d9b12f5b2', 'request_id': 'req-d4411bca-44dd-453b-9e82-24349fdb7403'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.792864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4053557239484ae5b823850d9b12f5b2', 'request_id': 'req-d4411bca-44dd-453b-9e82-24349fdb7403'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.805540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4053557239484ae5b823850d9b12f5b2', 'request_id': 'req-d4411bca-44dd-453b-9e82-24349fdb7403'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.827195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] The token cache is being invalidate because project 4053557239484ae5b823850d9b12f5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:34.827492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.828826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.833323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 997/2010] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:34 2022] DELETE /identity/v3/projects/4053557239484ae5b823850d9b12f5b2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:34.834878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:34.838574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:34.847532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:34.848442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.849103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.849529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.851883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:34.855033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:34.856523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4411bca-44dd-453b-9e82-24349fdb7403 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:34.866660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:34.867688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.870185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.870511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] PATH_INFO: `/v3/users/6f4993c486604dd882141fddee0674ec` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.872050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4993c486604dd882141fddee0674ec)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:34.876631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:34.876833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1014/2011] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:34.881874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:34.887561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:34.894121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:34.919556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b64b1-e334-40ec-a0c2-c5467942ab78 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:34.926164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:34.928266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:34.928870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:34.929902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] PATH_INFO: `/v3/projects/b08d0e5a9dd947a1ad5855b13ae89996/users/33002b8870a348dc92b6c34c0a6ff6d7/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:34.935900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4993c486604dd882141fddee0674ec'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.947871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4993c486604dd882141fddee0674ec'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.950060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08d0e5a9dd947a1ad5855b13ae89996, user_id=33002b8870a348dc92b6c34c0a6ff6d7, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:34.953381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:34.959672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4993c486604dd882141fddee0674ec'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.960448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4993c486604dd882141fddee0674ec'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.972657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1015/2012] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:34 2022] PUT /identity/v3/projects/b08d0e5a9dd947a1ad5855b13ae89996/users/33002b8870a348dc92b6c34c0a6ff6d7/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:34.976824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:34.978761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:34.980535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4993c486604dd882141fddee0674ec'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:34.987195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 998/2013] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:34 2022] DELETE /identity/v3/users/6f4993c486604dd882141fddee0674ec => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:35.053751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88018f0-9773-4f75-8e29-662caf8810e9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:35.126019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:35.134507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:35.136052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:35.136670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:35.137197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:35.138169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:35.140676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:35.157283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1016/2014] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:35.158459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:35.171461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce66ed-0665-47d7-91c0-ed5b16af2630 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:35.176895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aa1dc-4780-4162-8f63-243a71a5037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:35.178476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa1dc-4780-4162-8f63-243a71a5037c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:35.178771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa1dc-4780-4162-8f63-243a71a5037c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:35.179054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa1dc-4780-4162-8f63-243a71a5037c admin admin] PATH_INFO: `/v3/projects/b08d0e5a9dd947a1ad5855b13ae89996/users/33002b8870a348dc92b6c34c0a6ff6d7/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:35.183284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4aa1dc-4780-4162-8f63-243a71a5037c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08d0e5a9dd947a1ad5855b13ae89996, user_id=33002b8870a348dc92b6c34c0a6ff6d7, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:35.186629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4aa1dc-4780-4162-8f63-243a71a5037c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:35.202337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 999/2015] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:35 2022] PUT /identity/v3/projects/b08d0e5a9dd947a1ad5855b13ae89996/users/33002b8870a348dc92b6c34c0a6ff6d7/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:35.531143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:35.535480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:35.566882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf904d-4c6f-4c6b-ade3-3308769d6e8d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:35.671261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:35.672411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:35.675604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:35.676184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] PATH_INFO: `/v3/projects/a0cfe7a3b37742e38d8b7a0038a436e9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:35.678044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0cfe7a3b37742e38d8b7a0038a436e9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:35.681833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:35.823926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0cfe7a3b37742e38d8b7a0038a436e9', 'request_id': 'req-23caa495-ad28-415c-b27e-8f0248515e72'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:35.854526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0cfe7a3b37742e38d8b7a0038a436e9', 'request_id': 'req-23caa495-ad28-415c-b27e-8f0248515e72'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:35.867099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0cfe7a3b37742e38d8b7a0038a436e9', 'request_id': 'req-23caa495-ad28-415c-b27e-8f0248515e72'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:35.892345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] The token cache is being invalidate because project a0cfe7a3b37742e38d8b7a0038a436e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:35.892999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:35.895963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-23caa495-ad28-415c-b27e-8f0248515e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:35.897932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1017/2016] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:35 2022] DELETE /identity/v3/projects/a0cfe7a3b37742e38d8b7a0038a436e9 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:35.906234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2280a9-b111-4de9-9b94-530c315529be None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:35.906681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2280a9-b111-4de9-9b94-530c315529be None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:35.906779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2280a9-b111-4de9-9b94-530c315529be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:35.940743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7f2280a9-b111-4de9-9b94-530c315529be None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:35.947300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7f2280a9-b111-4de9-9b94-530c315529be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:36.103231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1000/2017] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:36.104652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421142-4c2e-431f-beb1-8d98be05642e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.104934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421142-4c2e-431f-beb1-8d98be05642e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.105253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421142-4c2e-431f-beb1-8d98be05642e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.145735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2d421142-4c2e-431f-beb1-8d98be05642e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:36.147004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2d421142-4c2e-431f-beb1-8d98be05642e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:36.171295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1018/2018] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:36.174993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2280a9-b111-4de9-9b94-530c315529be None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.199714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2280a9-b111-4de9-9b94-530c315529be None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.204833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.205707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.206104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.206442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.207381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:36.207817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.209542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.224462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1001/2019] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:36.228901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d421142-4c2e-431f-beb1-8d98be05642e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.238313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d421142-4c2e-431f-beb1-8d98be05642e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.243709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.244466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.244750 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.245021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.245732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.248403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.277011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1019/2020] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:36 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:36.281438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.295875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1032ff-a1c0-4168-91f1-e06a0564b586 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.377056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ede23-03e6-43dd-98da-378770bf7b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.377537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ede23-03e6-43dd-98da-378770bf7b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.377876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ede23-03e6-43dd-98da-378770bf7b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.378759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ede23-03e6-43dd-98da-378770bf7b80 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.379376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ede23-03e6-43dd-98da-378770bf7b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.381299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ede23-03e6-43dd-98da-378770bf7b80 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.400221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:36.409258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1002/2021] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:36 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:36.413199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.459500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf553c-329e-48aa-aaab-c9bdc5923be8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.460176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.460454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.460853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.466730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.467426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.467760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.468037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.468777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.470479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.483647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1020/2022] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:36.487080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.496417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] MFA Rules not processed for user `33002b8870a348dc92b6c34c0a6ff6d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:36.497991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:36.499231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea9981-da4b-4827-9dad-dc6b7970514a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.510972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.511881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.512494 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.512923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.514252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.517040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.543393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1021/2023] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:36.547606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.570683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f03c23-c5c9-4eea-80b1-f4317a08cd10 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.584408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c337aaa-441c-4771-a5c0-8921f7ffefc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.585477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c337aaa-441c-4771-a5c0-8921f7ffefc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.585768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c337aaa-441c-4771-a5c0-8921f7ffefc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.586068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c337aaa-441c-4771-a5c0-8921f7ffefc0 admin admin] PATH_INFO: `/v3/projects/848b1cc9c04e4e4981b0babf15699085/users/b08fac31cdda48be9d8fedc17d80e9e8/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.618058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c337aaa-441c-4771-a5c0-8921f7ffefc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=848b1cc9c04e4e4981b0babf15699085, user_id=b08fac31cdda48be9d8fedc17d80e9e8, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.622251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c337aaa-441c-4771-a5c0-8921f7ffefc0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.642475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1003/2024] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:36.650897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1022/2025] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:36 2022] PUT /identity/v3/projects/848b1cc9c04e4e4981b0babf15699085/users/b08fac31cdda48be9d8fedc17d80e9e8/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:36.657302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.659050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:36.668336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587cd94-5d01-472f-83b3-149fad642298 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.668683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587cd94-5d01-472f-83b3-149fad642298 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.668970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587cd94-5d01-472f-83b3-149fad642298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.708751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:36.736437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8587cd94-5d01-472f-83b3-149fad642298 None None] MFA Rules not processed for user `b08fac31cdda48be9d8fedc17d80e9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:36.737702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8587cd94-5d01-472f-83b3-149fad642298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:36.800508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0c8f7-d27d-46b3-851a-b6d01e5bbd8e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.845500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1023/2026] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:36.849924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587cd94-5d01-472f-83b3-149fad642298 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:36.875909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.877239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.877742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.878127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.882434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:36.941828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587cd94-5d01-472f-83b3-149fad642298 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:36.949671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:36.951163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:36.951627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:36.952336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:36.953346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.956350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.958877 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:36.965969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:36.967722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:36.988718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1004/2027] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:37.016280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:37.019412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1024/2028] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:36 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:37.033565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc396a-f37e-49f6-b539-ea6d29c9e543 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:37.178813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:37.180948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:37.193870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:37.194876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:37.195248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:37.195754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:37.196915 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:37.202634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:37.246617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1005/2029] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:37 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:37.250375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:37.270681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c873275-3d21-41c9-b30c-c693adc413da admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:37.288437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:37.289270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:37.289743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:37.290047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:37.291487 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:37.297603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:37.341346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745ac45-a3ae-4ba8-88f2-8bc8062645b1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:37.344890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1006/2030] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:37.347516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:37.353074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:37.356940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:37.358115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:37.358475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:37.365041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:37.370040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cce64-8da3-42e8-a0d8-771d3b35c6e9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:37.387720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:37.389731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:37.390662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:37.392155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] PATH_INFO: `/v3/projects/03161146ddd54179a09c8e048ded3a24/users/a314e9224bb445159be4842197bfd67f/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:37.457106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:37.462388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] RBAC: Authorizing `identity:create_grant(project_id=03161146ddd54179a09c8e048ded3a24, user_id=a314e9224bb445159be4842197bfd67f, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:37.472858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:37.507030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1007/2031] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:37 2022] PUT /identity/v3/projects/03161146ddd54179a09c8e048ded3a24/users/a314e9224bb445159be4842197bfd67f/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:37.512619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:37.515978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:37.517626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:37.547163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1025/2032] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:37.736476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fd12c-f10d-4683-b198-e8255cef827a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:37.750206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435dfbe3-aacf-4430-9fa8-e3faf2ef1388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:37.750879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dfbe3-aacf-4430-9fa8-e3faf2ef1388 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:37.751270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dfbe3-aacf-4430-9fa8-e3faf2ef1388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:37.751554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dfbe3-aacf-4430-9fa8-e3faf2ef1388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:37.752295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435dfbe3-aacf-4430-9fa8-e3faf2ef1388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:37.757210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435dfbe3-aacf-4430-9fa8-e3faf2ef1388 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:37.791078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1008/2033] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:37.793753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:37.807915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94221843-24b2-4acf-a796-528e1a24708e service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:37.822496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:37.838351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:37.838869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:37.839257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] PATH_INFO: `/v3/projects/03161146ddd54179a09c8e048ded3a24/users/a314e9224bb445159be4842197bfd67f/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:37.844818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03161146ddd54179a09c8e048ded3a24, user_id=a314e9224bb445159be4842197bfd67f, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:37.855575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:37.887123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1026/2034] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:37 2022] PUT /identity/v3/projects/03161146ddd54179a09c8e048ded3a24/users/a314e9224bb445159be4842197bfd67f/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:37.898380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a94619-7b30-416d-90c9-1ed80d6a74fe None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:37.898677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a94619-7b30-416d-90c9-1ed80d6a74fe None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:37.899011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a94619-7b30-416d-90c9-1ed80d6a74fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:37.974982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a2a94619-7b30-416d-90c9-1ed80d6a74fe None None] MFA Rules not processed for user `a314e9224bb445159be4842197bfd67f`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:37.980650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a2a94619-7b30-416d-90c9-1ed80d6a74fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:38.063674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:38.151658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1009/2035] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:38.178217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:38.322399 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d39172-b468-487b-a18d-4f04ebc69b5f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:38.326070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984dbd29-7f7f-486a-9706-0619647b899b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:38.326726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dbd29-7f7f-486a-9706-0619647b899b service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:38.327012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dbd29-7f7f-486a-9706-0619647b899b service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:38.327290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dbd29-7f7f-486a-9706-0619647b899b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:38.344910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dbd29-7f7f-486a-9706-0619647b899b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:38.346425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dbd29-7f7f-486a-9706-0619647b899b service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:38.365888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1027/2036] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:38.450579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a94619-7b30-416d-90c9-1ed80d6a74fe None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:38.558431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a94619-7b30-416d-90c9-1ed80d6a74fe None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:38.558431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:38.558431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:38.558686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:38.559691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] PATH_INFO: `/v3/users/b08fac31cdda48be9d8fedc17d80e9e8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:38.560111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08fac31cdda48be9d8fedc17d80e9e8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:38.562093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:38.648501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08fac31cdda48be9d8fedc17d80e9e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:38.657225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08fac31cdda48be9d8fedc17d80e9e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:38.668591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08fac31cdda48be9d8fedc17d80e9e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:38.669575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08fac31cdda48be9d8fedc17d80e9e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:38.688112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08fac31cdda48be9d8fedc17d80e9e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:38.696077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1010/2037] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:38 2022] DELETE /identity/v3/users/b08fac31cdda48be9d8fedc17d80e9e8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:38.701955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:38.702625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:38.702625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:38.722940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:38.724210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:38.800891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1028/2038] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:38.811456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:38.811994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:38.868433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:38.872154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ba05b-ba28-4ea6-b20d-ac0fad4c5e21 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:38.876468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:38.877535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:38.877931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:38.878222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:38.888628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:38.890105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:38.901546 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1011/2039] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:38.953410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:38.954962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:39.020324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51e874-093c-4d92-9a7c-854970ae7861 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:39.025333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:39.026075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:39.026335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:39.026895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:39.028754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:39.049650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:39.123604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:39.125092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:39.136003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1029/2040] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:39.212759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:39.214434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:39.237564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5c3a2-39b8-4a62-a501-f99e4de8a3c5 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:39.244380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:39.245347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:39.245702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:39.246380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] PATH_INFO: `/v3/projects/848b1cc9c04e4e4981b0babf15699085` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:39.247779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=848b1cc9c04e4e4981b0babf15699085)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:39.249968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:39.315556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848b1cc9c04e4e4981b0babf15699085', 'request_id': 'req-1128baa5-4ae9-4991-bc69-bde9e43724d2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.326147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848b1cc9c04e4e4981b0babf15699085', 'request_id': 'req-1128baa5-4ae9-4991-bc69-bde9e43724d2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.341147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848b1cc9c04e4e4981b0babf15699085', 'request_id': 'req-1128baa5-4ae9-4991-bc69-bde9e43724d2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.358910 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] The token cache is being invalidate because project 848b1cc9c04e4e4981b0babf15699085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:39.359492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.360177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.364782 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1012/2041] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:39 2022] DELETE /identity/v3/projects/848b1cc9c04e4e4981b0babf15699085 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:39.365980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:39.377920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db93cc-bb8f-4213-bec7-32deff7a2fbc service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:39.452573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:39.453854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:39.454509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:39.455336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] PATH_INFO: `/v3/users/a314e9224bb445159be4842197bfd67f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:39.456851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a314e9224bb445159be4842197bfd67f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:39.459258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:39.516654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a314e9224bb445159be4842197bfd67f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.530325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a314e9224bb445159be4842197bfd67f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.540516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a314e9224bb445159be4842197bfd67f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.541466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a314e9224bb445159be4842197bfd67f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.565795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7dd11673-1096-44b7-a965-1e5565fed576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a314e9224bb445159be4842197bfd67f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:39.586822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1030/2042] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:39 2022] DELETE /identity/v3/users/a314e9224bb445159be4842197bfd67f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:39.988420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:39.990254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:40.007047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128baa5-4ae9-4991-bc69-bde9e43724d2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.066861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.067740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.068017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.068298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] PATH_INFO: `/v3/projects/03161146ddd54179a09c8e048ded3a24` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.069595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] RBAC: Authorizing `identity:delete_project(project_id=03161146ddd54179a09c8e048ded3a24)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.072150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.120828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03161146ddd54179a09c8e048ded3a24', 'request_id': 'req-7005dd1b-5514-403f-84da-7e9e944fd78f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:40.128779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03161146ddd54179a09c8e048ded3a24', 'request_id': 'req-7005dd1b-5514-403f-84da-7e9e944fd78f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:40.135612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03161146ddd54179a09c8e048ded3a24', 'request_id': 'req-7005dd1b-5514-403f-84da-7e9e944fd78f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:40.146948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] The token cache is being invalidate because project 03161146ddd54179a09c8e048ded3a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:40.147237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:40.148444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7005dd1b-5514-403f-84da-7e9e944fd78f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:40.150358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1013/2043] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:39 2022] DELETE /identity/v3/projects/03161146ddd54179a09c8e048ded3a24 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:40.155212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b8af-7da5-4925-baf7-9d19f90cd27b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.155685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b8af-7da5-4925-baf7-9d19f90cd27b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.155863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b8af-7da5-4925-baf7-9d19f90cd27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.173931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4932b8af-7da5-4925-baf7-9d19f90cd27b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:40.175250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4932b8af-7da5-4925-baf7-9d19f90cd27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:40.207489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:40.241205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1031/2044] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:40.250501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d657bb-c1c2-4bd6-b656-4ae549982336 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.251030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d657bb-c1c2-4bd6-b656-4ae549982336 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.251249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d657bb-c1c2-4bd6-b656-4ae549982336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.262756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:40.278241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b3d657bb-c1c2-4bd6-b656-4ae549982336 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:40.281298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b3d657bb-c1c2-4bd6-b656-4ae549982336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:40.292006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1014/2045] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:40.296287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b8af-7da5-4925-baf7-9d19f90cd27b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.308289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b8af-7da5-4925-baf7-9d19f90cd27b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.313027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.313696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.314013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.314304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.315072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:40.315424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.317030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.331243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1032/2046] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:40.335894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d657bb-c1c2-4bd6-b656-4ae549982336 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.361551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d657bb-c1c2-4bd6-b656-4ae549982336 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.372101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.372978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.373384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.373581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.374363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.376576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.411525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1015/2047] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:40 2022] POST /identity/v3/projects => generated 399 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:40.415479 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.434554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0ab39-a0a7-442e-b05c-443b567da106 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.510012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.510706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.510992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.511403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.512140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.514115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.542275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1033/2048] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:40 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:40.546387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.559186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befa20e-3d27-49b9-ad86-f7212cd47d0c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.564036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.564736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.565051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.565329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.566109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.567736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.579510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1016/2049] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:40.583607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.595062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16855054-61aa-4aff-944e-b7be4adae150 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.600340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2028-be07-4423-96ed-0f3c524b19eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.601075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2028-be07-4423-96ed-0f3c524b19eb admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.601342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2028-be07-4423-96ed-0f3c524b19eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.601618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2028-be07-4423-96ed-0f3c524b19eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.602403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b2028-be07-4423-96ed-0f3c524b19eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.604738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b2028-be07-4423-96ed-0f3c524b19eb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.615561 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1034/2050] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:40.619846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.633947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7306b-2fa3-428b-82d0-2f9aa31055ef admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.635509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.636418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.636705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.637132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] PATH_INFO: `/v3/projects/d2f5dbbfcf664ffcb5872a5baa3daf20/users/5f8fe583eeeb4938bf7d0614bca02fb6/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.658652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f5dbbfcf664ffcb5872a5baa3daf20, user_id=5f8fe583eeeb4938bf7d0614bca02fb6, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.661665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.688752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1017/2051] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:40 2022] PUT /identity/v3/projects/d2f5dbbfcf664ffcb5872a5baa3daf20/users/5f8fe583eeeb4938bf7d0614bca02fb6/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:40.700968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d038b2-4fb5-4d15-b058-e9e1b39eb7d2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.701304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d038b2-4fb5-4d15-b058-e9e1b39eb7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.701589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d038b2-4fb5-4d15-b058-e9e1b39eb7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.739243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b7d038b2-4fb5-4d15-b058-e9e1b39eb7d2 None None] MFA Rules not processed for user `5f8fe583eeeb4938bf7d0614bca02fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:40.740920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b7d038b2-4fb5-4d15-b058-e9e1b39eb7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:40.833620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1035/2052] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:40.838741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.907793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ad14a-df63-49c2-b211-172ec550df5a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:40.914705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:40.916025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:40.916384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:40.916734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:40.917510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:40.920385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:40.955205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1018/2053] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:40 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:40.959338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d038b2-4fb5-4d15-b058-e9e1b39eb7d2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:40.971016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d038b2-4fb5-4d15-b058-e9e1b39eb7d2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:41.003737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:41.042870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:41.043555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.043839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.044119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.044844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:41.046845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:41.068161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1036/2054] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:40 2022] POST /identity/v3/users => generated 306 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:41.072731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:41.083126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cebf3-1b97-41eb-a2e7-377fb8706de6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:41.088530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:41.089198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.089484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.089763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.090530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:41.092132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:41.103130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1019/2055] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:41.106842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:41.116579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacc871-ee5f-417d-b4da-9eb84117499c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:41.121247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:41.122145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.122425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.122696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] PATH_INFO: `/v3/projects/81ed7982043a4bdaba4272b94060f60f/users/51f786091c8a4a01a29aa93081b57216/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.138881 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ed7982043a4bdaba4272b94060f60f, user_id=51f786091c8a4a01a29aa93081b57216, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:41.141552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:41.153759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1037/2056] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:41 2022] PUT /identity/v3/projects/81ed7982043a4bdaba4272b94060f60f/users/51f786091c8a4a01a29aa93081b57216/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:41.157674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:41.167747 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f72aa4-c6df-4586-9e93-d5ad51c61302 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:41.223097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de593850-0812-4fd8-9391-0c4ee2a7719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:41.223692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593850-0812-4fd8-9391-0c4ee2a7719b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.223977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593850-0812-4fd8-9391-0c4ee2a7719b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.224255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de593850-0812-4fd8-9391-0c4ee2a7719b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.224978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de593850-0812-4fd8-9391-0c4ee2a7719b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:41.226602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de593850-0812-4fd8-9391-0c4ee2a7719b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:41.236318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1020/2057] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:41.240379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:41.250621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e4165-bfd3-4a87-bc39-d07ee072076e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:41.255234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:41.256113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.256397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.256674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] PATH_INFO: `/v3/projects/81ed7982043a4bdaba4272b94060f60f/users/51f786091c8a4a01a29aa93081b57216/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.259205 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ed7982043a4bdaba4272b94060f60f, user_id=51f786091c8a4a01a29aa93081b57216, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:41.261831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:41.272031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1038/2058] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:41 2022] PUT /identity/v3/projects/81ed7982043a4bdaba4272b94060f60f/users/51f786091c8a4a01a29aa93081b57216/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:41.284638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ddae6-5c09-4ed3-95ad-131debec9ba5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.285177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ddae6-5c09-4ed3-95ad-131debec9ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.285497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ddae6-5c09-4ed3-95ad-131debec9ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.304781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:41.312799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c41ddae6-5c09-4ed3-95ad-131debec9ba5 None None] MFA Rules not processed for user `51f786091c8a4a01a29aa93081b57216`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:41.314357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c41ddae6-5c09-4ed3-95ad-131debec9ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:41.383303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1021/2059] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:41 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:41.394689 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:41.396415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:41.464737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84158-e771-4f4e-913e-88f61da33437 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:41.468495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:41.469144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.469423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.469725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.483822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:41.485316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:41.497013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1039/2060] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:41 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:41.569920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ddae6-5c09-4ed3-95ad-131debec9ba5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:41.580311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ddae6-5c09-4ed3-95ad-131debec9ba5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:41.584336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4431c-18f7-44d0-834a-b2c8ea9f27bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:41.584992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4431c-18f7-44d0-834a-b2c8ea9f27bd service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:41.585275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4431c-18f7-44d0-834a-b2c8ea9f27bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:41.585551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4431c-18f7-44d0-834a-b2c8ea9f27bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:41.650583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4431c-18f7-44d0-834a-b2c8ea9f27bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:41.652295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4431c-18f7-44d0-834a-b2c8ea9f27bd service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:41.666049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1022/2061] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:51.413252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:51.414797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:51.483569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8d33e-bded-40bc-b07d-ae533f6ead9b service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:51.488108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:51.488975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:51.489258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:51.489531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] PATH_INFO: `/v3/users/35c62b58564442e8a9c523e763ddd26f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:51.490743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c62b58564442e8a9c523e763ddd26f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:51.492704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:51.520060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:51.539930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c62b58564442e8a9c523e763ddd26f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:51.551918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c62b58564442e8a9c523e763ddd26f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:51.562510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c62b58564442e8a9c523e763ddd26f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:51.563173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c62b58564442e8a9c523e763ddd26f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:51.575676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c62b58564442e8a9c523e763ddd26f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:51.581736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1040/2062] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:51 2022] DELETE /identity/v3/users/35c62b58564442e8a9c523e763ddd26f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:51.586200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:51.586659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:51.586748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:51.605594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:51.606925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:51.673983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1023/2063] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:51.683935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:51.685367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:51.754488 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22cf8c-a26c-40de-aacc-70056b8ad8ed admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:51.758141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:51.758816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:51.759099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:51.759370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:51.769309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:51.770783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:51.781093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1041/2064] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:52.091137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:52.092760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:52.110276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead565-c99e-4f23-93d9-1a40c38d248c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:52.126713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:52.127627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:52.127912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:52.128260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] PATH_INFO: `/v3/projects/932c2e011cc249faa4b20aaed4f346fc` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:52.129668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=932c2e011cc249faa4b20aaed4f346fc)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:52.139764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:52.156203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:52.157763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:52.175265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c1096-69ff-4ad3-8ee6-97e847d920c9 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:52.180655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:52.182755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:52.184563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:52.184880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:52.185165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] PATH_INFO: `/v3/users/5f8fe583eeeb4938bf7d0614bca02fb6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:52.189033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8fe583eeeb4938bf7d0614bca02fb6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:52.193338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:52.198964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932c2e011cc249faa4b20aaed4f346fc', 'request_id': 'req-54942cb8-159f-4d6e-a74e-f941e9fde7ca'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.220496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932c2e011cc249faa4b20aaed4f346fc', 'request_id': 'req-54942cb8-159f-4d6e-a74e-f941e9fde7ca'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.231589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932c2e011cc249faa4b20aaed4f346fc', 'request_id': 'req-54942cb8-159f-4d6e-a74e-f941e9fde7ca'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.249776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] The token cache is being invalidate because project 932c2e011cc249faa4b20aaed4f346fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:52.250217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.250987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.252989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1024/2065] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:52 2022] DELETE /identity/v3/projects/932c2e011cc249faa4b20aaed4f346fc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:52.257400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:52.259107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:52.279131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8fe583eeeb4938bf7d0614bca02fb6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.297672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8fe583eeeb4938bf7d0614bca02fb6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.306593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8fe583eeeb4938bf7d0614bca02fb6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.307552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8fe583eeeb4938bf7d0614bca02fb6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.324705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dfdf0ad2-2a53-47c8-9a79-20c5f9a7c0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8fe583eeeb4938bf7d0614bca02fb6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.333890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1042/2066] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:52 2022] DELETE /identity/v3/users/5f8fe583eeeb4938bf7d0614bca02fb6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:52.337157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:52.337447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:52.338243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:52.355616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54942cb8-159f-4d6e-a74e-f941e9fde7ca admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:52.358175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:52.359500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:52.375751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:52.421369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:52.422460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:52.422825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:52.423036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] PATH_INFO: `/v3/users/c5e8e379c4dd4aaab569f3aa7d075787` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:52.424251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e8e379c4dd4aaab569f3aa7d075787)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:52.426522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:52.446354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1043/2067] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:52.456531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:52.462531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:52.488140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e8e379c4dd4aaab569f3aa7d075787'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.498627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e8e379c4dd4aaab569f3aa7d075787'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.509303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e8e379c4dd4aaab569f3aa7d075787'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.510422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e8e379c4dd4aaab569f3aa7d075787'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.517316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:52.533075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e8e379c4dd4aaab569f3aa7d075787'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:52.540165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4eadb-3746-4fbc-a2ee-6e58a2672fd4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:52.543919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1025/2068] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:52 2022] DELETE /identity/v3/users/c5e8e379c4dd4aaab569f3aa7d075787 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:52.547907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:52.549888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:52.550163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:52.550489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:52.553936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:52.583896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:52.590178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:52.602929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1044/2069] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:39:52.910704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:52.913185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.013574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba6a7c-9d53-494e-9cbd-31d26701621d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.018768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.019667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.019946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.020648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] PATH_INFO: `/v3/projects/cced49c028d242c2b4bea773f01e3581` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.022001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cced49c028d242c2b4bea773f01e3581)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.023931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.075857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.078595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.098017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e682b-22b8-4ccb-8962-fed337d6d4e6 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.113484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cced49c028d242c2b4bea773f01e3581', 'request_id': 'req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.124694 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cced49c028d242c2b4bea773f01e3581', 'request_id': 'req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.132005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cced49c028d242c2b4bea773f01e3581', 'request_id': 'req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.146035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] The token cache is being invalidate because project cced49c028d242c2b4bea773f01e3581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:53.146288 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.147069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4cb7b37c-2fd0-48cb-8095-6c7874547b7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.148690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1026/2070] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:52 2022] DELETE /identity/v3/projects/cced49c028d242c2b4bea773f01e3581 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:53.155431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0023-c9ff-4507-be9c-48fe5358a0ee None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.155597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0023-c9ff-4507-be9c-48fe5358a0ee None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.155846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0023-c9ff-4507-be9c-48fe5358a0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.176394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4c7b0023-c9ff-4507-be9c-48fe5358a0ee None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:53.177640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4c7b0023-c9ff-4507-be9c-48fe5358a0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.182058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.182974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.183264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.183544 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] PATH_INFO: `/v3/projects/d2f5dbbfcf664ffcb5872a5baa3daf20` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.184738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f5dbbfcf664ffcb5872a5baa3daf20)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.186531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1027/2071] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:53.194820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.201591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023cc52-781a-4b54-97f8-010121556d98 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.202684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023cc52-781a-4b54-97f8-010121556d98 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.203241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023cc52-781a-4b54-97f8-010121556d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.231655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8023cc52-781a-4b54-97f8-010121556d98 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:53.232983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8023cc52-781a-4b54-97f8-010121556d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.245284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1028/2072] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:53.249639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023cc52-781a-4b54-97f8-010121556d98 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.262331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023cc52-781a-4b54-97f8-010121556d98 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.266580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.267273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.267562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.267841 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.268903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:53.269217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.271072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.285052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1029/2073] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:53.290104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.306492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d74f-b2c4-4568-b0d7-04f4f29b9669 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.306696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f5dbbfcf664ffcb5872a5baa3daf20', 'request_id': 'req-521ab4fc-e8d3-486b-aedd-dbced9702c1f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.317472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f5dbbfcf664ffcb5872a5baa3daf20', 'request_id': 'req-521ab4fc-e8d3-486b-aedd-dbced9702c1f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.325328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:53.327474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f5dbbfcf664ffcb5872a5baa3daf20', 'request_id': 'req-521ab4fc-e8d3-486b-aedd-dbced9702c1f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.354613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] The token cache is being invalidate because project d2f5dbbfcf664ffcb5872a5baa3daf20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:53.354887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.355629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.358337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1045/2074] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:53 2022] DELETE /identity/v3/projects/d2f5dbbfcf664ffcb5872a5baa3daf20 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:53.361981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.363512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.381138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:53.402568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.403807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.404260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.405455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.406474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.409059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.442295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ab4fc-e8d3-486b-aedd-dbced9702c1f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.444363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1030/2075] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:53 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:53.448556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.450929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.502596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.503543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.503828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.504209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] PATH_INFO: `/v3/users/51f786091c8a4a01a29aa93081b57216` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.506356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f786091c8a4a01a29aa93081b57216)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.508329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.521416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd167fc1-acf7-4414-be39-07b344da8fc1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.526352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.527053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.527356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.527630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.528320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.530264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.551270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1031/2076] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:53 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:53.555201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.557313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f786091c8a4a01a29aa93081b57216'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.566177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aa2c5-2969-4c86-a0fc-9c3767ea4714 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.567556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f786091c8a4a01a29aa93081b57216'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.571060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.571772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.572064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.572344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.573077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.575264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.575414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f786091c8a4a01a29aa93081b57216'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.576060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f786091c8a4a01a29aa93081b57216'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.585410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1032/2077] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:53.589072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f786091c8a4a01a29aa93081b57216'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:53.589224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.590815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.595217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1046/2078] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:53 2022] DELETE /identity/v3/users/51f786091c8a4a01a29aa93081b57216 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:53.603996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b37df1-1a92-494b-8847-727e57bb3e88 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.691538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71b10-37a2-41f8-bc2b-0f5b8bb0eb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.692744 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71b10-37a2-41f8-bc2b-0f5b8bb0eb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.693323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71b10-37a2-41f8-bc2b-0f5b8bb0eb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.693937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71b10-37a2-41f8-bc2b-0f5b8bb0eb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.703365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71b10-37a2-41f8-bc2b-0f5b8bb0eb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.705789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71b10-37a2-41f8-bc2b-0f5b8bb0eb07 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.721601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1033/2079] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:53.726538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:53.742785 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f04fe8-d3af-465d-b911-b11e55302c80 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:53.748373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:53.749290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.749575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.749880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] PATH_INFO: `/v3/projects/b1d603761e5c42818bc164715942dabf/users/1b234ecbd802498ea6434b5e882ac929/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.769477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d603761e5c42818bc164715942dabf, user_id=1b234ecbd802498ea6434b5e882ac929, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:53.772208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:53.774757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:53.790526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1047/2080] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:53 2022] PUT /identity/v3/projects/b1d603761e5c42818bc164715942dabf/users/1b234ecbd802498ea6434b5e882ac929/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:53.804034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:53.804282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:53.804570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:53.847926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] MFA Rules not processed for user `1b234ecbd802498ea6434b5e882ac929`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:53.849367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:53.903219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:53.936303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1034/2081] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:53.940834 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.016955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefdedb-6b9e-4e8a-98a2-869106b4e033 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.023409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.024195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.024480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.024764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.025458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.028029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.054640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.056326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:54.062465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1048/2082] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:53 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:54.063756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.076639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353738e-7483-4cef-9661-db2faf8d839c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.130386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed095-7df6-4092-be63-1953f3903d95 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.134987 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.135665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.135976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.136260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.136960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.137119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.138136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.138491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.138834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] PATH_INFO: `/v3/projects/81ed7982043a4bdaba4272b94060f60f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.138984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.140157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ed7982043a4bdaba4272b94060f60f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.142270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.161095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1049/2083] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:54 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:54.165393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.178384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1ff98-75ac-49b3-aa71-c50ebed7fb91 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.186278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.186962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.187249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.187528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.188268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.189930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.198173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ed7982043a4bdaba4272b94060f60f', 'request_id': 'req-cafec73d-b137-4b23-abeb-a3927c212ec7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:54.200912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1050/2084] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:54.205792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.206449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ed7982043a4bdaba4272b94060f60f', 'request_id': 'req-cafec73d-b137-4b23-abeb-a3927c212ec7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:54.212493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ed7982043a4bdaba4272b94060f60f', 'request_id': 'req-cafec73d-b137-4b23-abeb-a3927c212ec7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:54.217114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350026-fd57-436e-b165-2763eaacd4aa admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.225733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] The token cache is being invalidate because project 81ed7982043a4bdaba4272b94060f60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:54.226062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:54.226724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cafec73d-b137-4b23-abeb-a3927c212ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:54.228278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1035/2085] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:54 2022] DELETE /identity/v3/projects/81ed7982043a4bdaba4272b94060f60f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:54.233563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.233881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.234154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.252292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:54.253504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:54.271006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:54.280235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f89e-2c76-4383-8dac-f830b3e17a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.281522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f89e-2c76-4383-8dac-f830b3e17a03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.281817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f89e-2c76-4383-8dac-f830b3e17a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.282161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f89e-2c76-4383-8dac-f830b3e17a03 admin admin] PATH_INFO: `/v3/projects/c3e8587fab7a41b9a4f4364a9120e278/users/0061667ea53f4af79350efecdd72a52f/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.298520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4f89e-2c76-4383-8dac-f830b3e17a03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e8587fab7a41b9a4f4364a9120e278, user_id=0061667ea53f4af79350efecdd72a52f, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.301799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4f89e-2c76-4383-8dac-f830b3e17a03 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.318951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1036/2086] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:54.320569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1051/2087] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:54 2022] PUT /identity/v3/projects/c3e8587fab7a41b9a4f4364a9120e278/users/0061667ea53f4af79350efecdd72a52f/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:54.324559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.326406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:54.326653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e6183-79f4-4c70-9fa5-d51532e99b22 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.327006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e6183-79f4-4c70-9fa5-d51532e99b22 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.327381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e6183-79f4-4c70-9fa5-d51532e99b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.339956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5dee7-2b59-41d0-b09b-3ade729cf67b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.348863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4e5e6183-79f4-4c70-9fa5-d51532e99b22 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:54.349015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:54.349483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4e5e6183-79f4-4c70-9fa5-d51532e99b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:54.409400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.410260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.410644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.411021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.412017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.413961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.424862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1037/2088] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:54.429291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.436744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1052/2089] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:54.440611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e6183-79f4-4c70-9fa5-d51532e99b22 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.445015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059e4c6-604f-4a19-8325-2952eb19855e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.452250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17da52-6b67-45f1-8c95-242616f1b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.453911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17da52-6b67-45f1-8c95-242616f1b161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.454202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17da52-6b67-45f1-8c95-242616f1b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.454480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17da52-6b67-45f1-8c95-242616f1b161 admin admin] PATH_INFO: `/v3/projects/c3e8587fab7a41b9a4f4364a9120e278/users/0061667ea53f4af79350efecdd72a52f/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.456684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17da52-6b67-45f1-8c95-242616f1b161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e8587fab7a41b9a4f4364a9120e278, user_id=0061667ea53f4af79350efecdd72a52f, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.460189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17da52-6b67-45f1-8c95-242616f1b161 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.488128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1038/2090] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:54 2022] PUT /identity/v3/projects/c3e8587fab7a41b9a4f4364a9120e278/users/0061667ea53f4af79350efecdd72a52f/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:54.494460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e6183-79f4-4c70-9fa5-d51532e99b22 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.495125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.495544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.497386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.562448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] MFA Rules not processed for user `0061667ea53f4af79350efecdd72a52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:54.564359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:54.622874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.623748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.624051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.624382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.625303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:54.625665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.631939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.661281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1053/2091] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:39:54.661281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.675301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fb328-c528-444a-955c-ca89422fe050 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.687012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.687740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.688025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.688304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.689001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.691839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.696573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1039/2092] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:54.710605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.712382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:54.742147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1054/2093] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:54 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:54.742147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.788469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23095b9-c0e7-435c-9b26-482ddd5429c4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.800806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:54.813968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dda8a-b073-4b28-bfdc-68a2d33d3e32 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.819309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.820006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.820292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.820570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.821396 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.823374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.826826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:54.848923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1055/2094] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:54 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:54.851321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a81cd-dd62-4fec-8c5f-f51c42ed7c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.851996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a81cd-dd62-4fec-8c5f-f51c42ed7c44 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.852417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a81cd-dd62-4fec-8c5f-f51c42ed7c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.852607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a81cd-dd62-4fec-8c5f-f51c42ed7c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.853264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.855469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-113a81cd-dd62-4fec-8c5f-f51c42ed7c44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:54.864359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f356337-2d41-486c-80f0-3d152355017c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.869320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.870265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.870776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.871168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.871981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.873775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.890634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1056/2095] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:54.892167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.903717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec66a0-88ba-4bda-8ac2-e18113cb4a00 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.909300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.910276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.910564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.910843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.911640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.913309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.925766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1057/2096] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:54.929976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:54.941325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a81cd-dd62-4fec-8c5f-f51c42ed7c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.941866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcea14-db54-47df-9dda-b0ee598d2ede admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:54.943017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a81cd-dd62-4fec-8c5f-f51c42ed7c44 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.947642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:54.949105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:54.949460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:54.950509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] PATH_INFO: `/v3/projects/b63fbd5597ef4fb9b14e40bd7b08f269/users/5b4ac88992b7491ca26e9876d96728b1/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:54.966959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1040/2097] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:54.972149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63fbd5597ef4fb9b14e40bd7b08f269, user_id=5b4ac88992b7491ca26e9876d96728b1, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:54.974791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:54.998633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1058/2098] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:54 2022] PUT /identity/v3/projects/b63fbd5597ef4fb9b14e40bd7b08f269/users/5b4ac88992b7491ca26e9876d96728b1/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:39:55.009697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8d2a6-e6f6-4448-b139-4b5268996534 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.010020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8d2a6-e6f6-4448-b139-4b5268996534 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.010306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8d2a6-e6f6-4448-b139-4b5268996534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.074160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9bb8d2a6-e6f6-4448-b139-4b5268996534 None None] MFA Rules not processed for user `5b4ac88992b7491ca26e9876d96728b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:55.075679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9bb8d2a6-e6f6-4448-b139-4b5268996534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:55.195647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1041/2099] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:55.200577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:55.327825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8738fdf-df43-471f-8690-3da65f594819 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:55.341924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:55.343059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.343405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.343668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.344588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:55.350570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:55.428824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1059/2100] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:55 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:55.434474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8d2a6-e6f6-4448-b139-4b5268996534 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:55.459412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8d2a6-e6f6-4448-b139-4b5268996534 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:55.519564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:55.520359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.520787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.521068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.521777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:55.523739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:55.549763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1042/2101] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:55 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:55.550572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:55.563125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b95ba-17c0-467f-a501-55df7a7f569e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:55.567438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:55.568297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.568951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.568951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.569572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:55.571198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:55.584325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1060/2102] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:55.588457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:55.599194 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979264f-2573-443c-8eb9-0ddf58e29a06 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:55.603968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:55.604866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.605152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.605430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] PATH_INFO: `/v3/projects/e8033f545ba84ab4acc25496d794b500/users/2875953af2ac488d96d3dbe96b0a12f4/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.607930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:55.628193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8033f545ba84ab4acc25496d794b500, user_id=2875953af2ac488d96d3dbe96b0a12f4, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:55.631359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:55.646414 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1043/2103] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:55 2022] PUT /identity/v3/projects/e8033f545ba84ab4acc25496d794b500/users/2875953af2ac488d96d3dbe96b0a12f4/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:55.650564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:55.662137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a33bc2-41e7-449d-bd68-a8dbd78f1b03 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:55.721498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88284d9e-5079-4915-9804-17a174180546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:55.722210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284d9e-5079-4915-9804-17a174180546 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.722497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284d9e-5079-4915-9804-17a174180546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.722927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284d9e-5079-4915-9804-17a174180546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.723661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284d9e-5079-4915-9804-17a174180546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:55.725259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284d9e-5079-4915-9804-17a174180546 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:55.739509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1061/2104] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:55.744264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:55.756245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e726c-24ab-4682-83fe-91b779f9cd39 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:55.761685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535b080-482d-4545-b803-366e66630807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:55.763490 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b080-482d-4545-b803-366e66630807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.763924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b080-482d-4545-b803-366e66630807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.764491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b080-482d-4545-b803-366e66630807 admin admin] PATH_INFO: `/v3/projects/e8033f545ba84ab4acc25496d794b500/users/2875953af2ac488d96d3dbe96b0a12f4/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.767272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535b080-482d-4545-b803-366e66630807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8033f545ba84ab4acc25496d794b500, user_id=2875953af2ac488d96d3dbe96b0a12f4, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:55.769979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535b080-482d-4545-b803-366e66630807 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:55.782794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1044/2105] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:55 2022] PUT /identity/v3/projects/e8033f545ba84ab4acc25496d794b500/users/2875953af2ac488d96d3dbe96b0a12f4/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:55.794467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b9efb-f990-44b9-b14b-49802a93d253 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:55.794779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b9efb-f990-44b9-b14b-49802a93d253 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:55.795038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b9efb-f990-44b9-b14b-49802a93d253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:55.818653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:55.838404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b73b9efb-f990-44b9-b14b-49802a93d253 None None] MFA Rules not processed for user `2875953af2ac488d96d3dbe96b0a12f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:55.839753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b73b9efb-f990-44b9-b14b-49802a93d253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:55.929674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1062/2106] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:55.944168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535b080-482d-4545-b803-366e66630807 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:55.946418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1535b080-482d-4545-b803-366e66630807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:56.008337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535b080-482d-4545-b803-366e66630807 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:56.013179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d50e1-147a-4062-838f-4c42c7a6448e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:56.014044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d50e1-147a-4062-838f-4c42c7a6448e service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:56.014249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d50e1-147a-4062-838f-4c42c7a6448e service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:56.014529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d50e1-147a-4062-838f-4c42c7a6448e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:56.025212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d50e1-147a-4062-838f-4c42c7a6448e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:56.027262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d50e1-147a-4062-838f-4c42c7a6448e service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:56.037538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1045/2107] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:57.191966 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b9efb-f990-44b9-b14b-49802a93d253 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:57.261748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b9efb-f990-44b9-b14b-49802a93d253 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:57.267556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:57.268670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:57.268859 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:57.269196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] PATH_INFO: `/v3/users/5b4ac88992b7491ca26e9876d96728b1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:57.270788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4ac88992b7491ca26e9876d96728b1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:57.272952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:57.345670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4ac88992b7491ca26e9876d96728b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.356498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4ac88992b7491ca26e9876d96728b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.364297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4ac88992b7491ca26e9876d96728b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.365270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4ac88992b7491ca26e9876d96728b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.377055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4ac88992b7491ca26e9876d96728b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.383044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1063/2108] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:57 2022] DELETE /identity/v3/users/5b4ac88992b7491ca26e9876d96728b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:57.388518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541615ab-7674-4cb5-b18a-511438247192 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:57.388967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541615ab-7674-4cb5-b18a-511438247192 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:57.389199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541615ab-7674-4cb5-b18a-511438247192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:57.412767 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-541615ab-7674-4cb5-b18a-511438247192 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:57.414179 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-541615ab-7674-4cb5-b18a-511438247192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:57.481894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1046/2109] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:57.490368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:57.491764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:57.505699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:57.554401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a828df-06c6-4322-9acd-91bd544d3552 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:57.558796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:57.559441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:57.559721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:57.559993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:57.569334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:57.570870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:57.580891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1064/2110] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:57.891468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541615ab-7674-4cb5-b18a-511438247192 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:57.893827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-541615ab-7674-4cb5-b18a-511438247192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:57.909251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541615ab-7674-4cb5-b18a-511438247192 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:57.915461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:57.915905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:57.916803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:57.917176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] PATH_INFO: `/v3/projects/b63fbd5597ef4fb9b14e40bd7b08f269` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:57.919431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63fbd5597ef4fb9b14e40bd7b08f269)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:57.921633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:57.972315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63fbd5597ef4fb9b14e40bd7b08f269', 'request_id': 'req-4083dffc-673f-40e3-8a59-117d86e1dd30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.980262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63fbd5597ef4fb9b14e40bd7b08f269', 'request_id': 'req-4083dffc-673f-40e3-8a59-117d86e1dd30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.986643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63fbd5597ef4fb9b14e40bd7b08f269', 'request_id': 'req-4083dffc-673f-40e3-8a59-117d86e1dd30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:57.988611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:58.001561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] The token cache is being invalidate because project b63fbd5597ef4fb9b14e40bd7b08f269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:58.001868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.002522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.004148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1047/2111] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:57 2022] DELETE /identity/v3/projects/b63fbd5597ef4fb9b14e40bd7b08f269 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:58.008395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:58.019405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3b019-cfc5-49e9-ae34-99e8c89a09ef service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:58.076761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:58.077623 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:58.077935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:58.078212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] PATH_INFO: `/v3/users/2875953af2ac488d96d3dbe96b0a12f4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:58.079382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2875953af2ac488d96d3dbe96b0a12f4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:58.081283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:58.129538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2875953af2ac488d96d3dbe96b0a12f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.137588 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2875953af2ac488d96d3dbe96b0a12f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.145134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2875953af2ac488d96d3dbe96b0a12f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.146041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2875953af2ac488d96d3dbe96b0a12f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.160385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac81c9ea-f413-4272-a277-01229d1c255d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2875953af2ac488d96d3dbe96b0a12f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.166506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1065/2112] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:58 2022] DELETE /identity/v3/users/2875953af2ac488d96d3dbe96b0a12f4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:58.555642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:58.556997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:58.573235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083dffc-673f-40e3-8a59-117d86e1dd30 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:58.644418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:58.645296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:58.645583 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:58.645890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] PATH_INFO: `/v3/projects/e8033f545ba84ab4acc25496d794b500` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:58.647091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8033f545ba84ab4acc25496d794b500)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:58.649002 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:58.706162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8033f545ba84ab4acc25496d794b500', 'request_id': 'req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.714106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8033f545ba84ab4acc25496d794b500', 'request_id': 'req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.720470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8033f545ba84ab4acc25496d794b500', 'request_id': 'req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.738028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] The token cache is being invalidate because project e8033f545ba84ab4acc25496d794b500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:39:58.738322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.738966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-87e14d7c-b0e4-4ed1-9fd9-32b65ff07640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:58.741770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1048/2113] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:39:58 2022] DELETE /identity/v3/projects/e8033f545ba84ab4acc25496d794b500 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:58.747059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878b6d-4283-4c28-8276-4db1acfa76db None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:58.747571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878b6d-4283-4c28-8276-4db1acfa76db None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:58.747664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35878b6d-4283-4c28-8276-4db1acfa76db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:58.766532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-35878b6d-4283-4c28-8276-4db1acfa76db None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:58.768020 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-35878b6d-4283-4c28-8276-4db1acfa76db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:58.798810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:58.835112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1066/2114] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:58.842748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42dc31-0155-4a93-9803-d0ee20c9205f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:58.843035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42dc31-0155-4a93-9803-d0ee20c9205f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:58.843313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42dc31-0155-4a93-9803-d0ee20c9205f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:58.869011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be42dc31-0155-4a93-9803-d0ee20c9205f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:58.870914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be42dc31-0155-4a93-9803-d0ee20c9205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:58.882797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1049/2115] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:58.887437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35878b6d-4283-4c28-8276-4db1acfa76db None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:58.898049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35878b6d-4283-4c28-8276-4db1acfa76db None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:58.902842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:58.903510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:58.903799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:58.904080 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:58.904818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:39:58.905133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:58.906779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:58.922997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1067/2116] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:39:58.924125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42dc31-0155-4a93-9803-d0ee20c9205f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:58.938322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42dc31-0155-4a93-9803-d0ee20c9205f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:58.944314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:58.945015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:58.945292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:58.945574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:58.946342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:58.948322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:58.975245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1050/2117] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:58 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:39:58.979009 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:58.989851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25caac4-bb67-4ee2-842f-bbfaaea2e1a8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.050141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.050808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.051160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.051461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.052356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.054424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.080837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1068/2118] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:58 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:59.085728 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.087049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.089293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:59.097963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c559cf-0b97-4407-bfa1-e68eb5439cab admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.103107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.103823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.104156 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.104427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.105712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.107359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.113279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e6ff4-a894-43a4-b5a5-30161cf3b71a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.119029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.120142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1051/2119] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:59.122316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.122605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.122885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] PATH_INFO: `/v3/users/1b234ecbd802498ea6434b5e882ac929` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.124316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b234ecbd802498ea6434b5e882ac929)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.125429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.126387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.136362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d748f-4dc7-408e-8cac-b80aab81a3f1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.141346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.142036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.142521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.142649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.143338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.145462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.157874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1052/2120] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:39:59.161504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.174212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22019cd4-a197-451b-a91c-db67cae09246 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.179936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e7d9e-bd26-4b7f-a20c-21d2ad898c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.180943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e7d9e-bd26-4b7f-a20c-21d2ad898c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.181292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e7d9e-bd26-4b7f-a20c-21d2ad898c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.181636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e7d9e-bd26-4b7f-a20c-21d2ad898c35 admin admin] PATH_INFO: `/v3/projects/d41282deaa8e415893d84f60831e737c/users/c8c4d7e6692142908031e4b05042707a/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.189507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b234ecbd802498ea6434b5e882ac929'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:59.200033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b234ecbd802498ea6434b5e882ac929'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:59.203402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e7d9e-bd26-4b7f-a20c-21d2ad898c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41282deaa8e415893d84f60831e737c, user_id=c8c4d7e6692142908031e4b05042707a, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.207132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e7d9e-bd26-4b7f-a20c-21d2ad898c35 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.208498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b234ecbd802498ea6434b5e882ac929'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:59.209639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b234ecbd802498ea6434b5e882ac929'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:59.223495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1053/2121] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:59 2022] PUT /identity/v3/projects/d41282deaa8e415893d84f60831e737c/users/c8c4d7e6692142908031e4b05042707a/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:59.239374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.240286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.240773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.250077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1d22c9d4-8332-4617-bf30-c7daa1c7a30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b234ecbd802498ea6434b5e882ac929'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:39:59.258420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1069/2122] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:39:59 2022] DELETE /identity/v3/users/1b234ecbd802498ea6434b5e882ac929 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:59.262925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.263215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.263489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.268603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:59.282765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] MFA Rules not processed for user `c8c4d7e6692142908031e4b05042707a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:59.283510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:59.284359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:59.285477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:59.309260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:39:59.372261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1070/2123] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:39:59.383389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.387023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:59.389189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1054/2124] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:39:59.393657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.395885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:59.425518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2823f-2f18-4474-95df-93bfbcb19064 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.433896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.434654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.435022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.435826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.436667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.438887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.500621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1055/2125] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:39:59 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:59.504352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.517740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dfa1-1937-4b4f-8bd2-7a5a83aa838b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.604073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d21b0f-8cfd-455e-9f4c-14dd05f32678 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.615632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.616927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.617178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.622946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.623809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.629119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.629119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.629119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.629119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.629512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.647257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.649883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.661113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1056/2126] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:39:59 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:39:59.665070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1071/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:39:59.666921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.680007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b193746-9bb1-4405-98a0-13ef882f3f6e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.686178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.687074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.687532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.687833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.688577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.690276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.706171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1057/2128] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:59.708558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.721459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbbc47-4964-4d13-aac7-bbf3f2e9c316 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.727939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4ca44-6ff4-4a64-9ef9-64792d6511f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.728774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4ca44-6ff4-4a64-9ef9-64792d6511f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.729053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4ca44-6ff4-4a64-9ef9-64792d6511f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.729813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4ca44-6ff4-4a64-9ef9-64792d6511f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.730121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4ca44-6ff4-4a64-9ef9-64792d6511f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.731817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4ca44-6ff4-4a64-9ef9-64792d6511f4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.743515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1072/2129] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:39:59.752077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:39:59.768041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c21e2f-ba68-44ab-9d91-e1a7a4754fbd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:39:59.781634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:39:59.782577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.782989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.783299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] PATH_INFO: `/v3/projects/7283130273b245e782269b94cf6638a1/users/a2b1c8fbee524103a9c91023c774f340/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.808731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7283130273b245e782269b94cf6638a1, user_id=a2b1c8fbee524103a9c91023c774f340, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:39:59.812296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:39:59.854015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1058/2130] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:39:59 2022] PUT /identity/v3/projects/7283130273b245e782269b94cf6638a1/users/a2b1c8fbee524103a9c91023c774f340/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:39:59.857532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:39:59.858066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:39:59.858405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:39:59.892650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] MFA Rules not processed for user `a2b1c8fbee524103a9c91023c774f340`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:39:59.892990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:39:59.996898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:00.007951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1073/2131] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:39:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:00.013350 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.069819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:00.092434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.094018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:00.106548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8677-d02a-4983-a18f-591af0384518 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.113284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.114304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.114687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.115061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.116249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.118146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.119109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687241c-4ed9-44cb-b544-f9edf65a64b6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.139539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.140705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.141135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.141654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] PATH_INFO: `/v3/projects/b1d603761e5c42818bc164715942dabf` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.144324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d603761e5c42818bc164715942dabf)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.147216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.153676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1059/2132] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:00 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:00.157241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.214263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d603761e5c42818bc164715942dabf', 'request_id': 'req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.237548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d603761e5c42818bc164715942dabf', 'request_id': 'req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.246064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ebcba-bbeb-4d73-80ad-98f53d31a763 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.249772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d603761e5c42818bc164715942dabf', 'request_id': 'req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.266864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] The token cache is being invalidate because project b1d603761e5c42818bc164715942dabf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:00.267373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.268328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.271267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1074/2133] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:00 2022] DELETE /identity/v3/projects/b1d603761e5c42818bc164715942dabf => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:40:00.275453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.278912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:00.335647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.336317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.336622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.336934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.337765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.340801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.375027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1060/2134] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:00 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:00.375197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cf80b-61a3-4c20-9a44-b3ef8d7eaf38 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.376549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.380839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:00.382092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.383641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.383923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.384243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] PATH_INFO: `/v3/users/0061667ea53f4af79350efecdd72a52f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.385660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0061667ea53f4af79350efecdd72a52f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.388227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.394384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40570da-3327-473d-9367-52b63cf21887 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.399382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.400066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.400361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.400645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.401574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.403434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.415730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1061/2135] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:00.420697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.433756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7f450-0bb3-4696-a8f4-2c2f058bacce admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.440510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.441858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.442323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.442698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] PATH_INFO: `/v3/projects/0c50f01fb0cf4a948a4923347834013d/users/6c991f70e7b44a129d9c163e8302ab69/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.453486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0061667ea53f4af79350efecdd72a52f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.460306 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c50f01fb0cf4a948a4923347834013d, user_id=6c991f70e7b44a129d9c163e8302ab69, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.464583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0061667ea53f4af79350efecdd72a52f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.470510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.484718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0061667ea53f4af79350efecdd72a52f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.485830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0061667ea53f4af79350efecdd72a52f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.493190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1062/2136] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:00 2022] PUT /identity/v3/projects/0c50f01fb0cf4a948a4923347834013d/users/6c991f70e7b44a129d9c163e8302ab69/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:40:00.497904 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.499651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:00.510204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0061667ea53f4af79350efecdd72a52f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.520925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1075/2137] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:00 2022] DELETE /identity/v3/users/0061667ea53f4af79350efecdd72a52f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:00.529438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:00.589581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62071445-e154-462c-99e2-d9e4c8fc5a99 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.663725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.664575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.665462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.665882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.666911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.670080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.690566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.692159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1063/2138] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:00.701710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:00.708643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266251ff-e91b-41ce-8d14-878282dc0b28 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.719666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.722102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:00.722645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1372ca-1384-4a3f-8ed4-a1f0a2b95d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.723599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1372ca-1384-4a3f-8ed4-a1f0a2b95d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.723884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1372ca-1384-4a3f-8ed4-a1f0a2b95d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.724161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1372ca-1384-4a3f-8ed4-a1f0a2b95d11 admin admin] PATH_INFO: `/v3/projects/0c50f01fb0cf4a948a4923347834013d/users/6c991f70e7b44a129d9c163e8302ab69/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.727572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1372ca-1384-4a3f-8ed4-a1f0a2b95d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c50f01fb0cf4a948a4923347834013d, user_id=6c991f70e7b44a129d9c163e8302ab69, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.730926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1372ca-1384-4a3f-8ed4-a1f0a2b95d11 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.740686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652f9fb-fc54-44ec-ade5-925b9ded87e8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.744097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1076/2139] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:00 2022] PUT /identity/v3/projects/0c50f01fb0cf4a948a4923347834013d/users/6c991f70e7b44a129d9c163e8302ab69/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:00.757742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.758047 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.758355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.794707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] MFA Rules not processed for user `6c991f70e7b44a129d9c163e8302ab69`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:00.795943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:00.825637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.828567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.828962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.829349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] PATH_INFO: `/v3/users/6c60de8fd0c64a44bb71c6e98e7ede9a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.831215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c60de8fd0c64a44bb71c6e98e7ede9a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:00.834581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:00.881688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1077/2140] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:00.894487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:00.896044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:00.914926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c60de8fd0c64a44bb71c6e98e7ede9a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.930169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c60de8fd0c64a44bb71c6e98e7ede9a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.943475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c60de8fd0c64a44bb71c6e98e7ede9a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.947752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c60de8fd0c64a44bb71c6e98e7ede9a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.954453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:00.978196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb72d2-b9ee-4c00-ab08-3c4c69f659ec None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:00.984566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c60de8fd0c64a44bb71c6e98e7ede9a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:00.988805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:00.989609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:00.990069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:00.990451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:00.992981 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1064/2141] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:00 2022] DELETE /identity/v3/users/6c60de8fd0c64a44bb71c6e98e7ede9a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:00.997167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:01.076288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:01.077883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:01.087908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1078/2142] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:40:01.169879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:01.171314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:01.294099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:01.299013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:01.302017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1482d32-726b-4bb2-bfb9-a7f27634f5c8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:01.318522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:01.319595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.322296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.322659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] PATH_INFO: `/v3/projects/c3e8587fab7a41b9a4f4364a9120e278` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.331004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e8587fab7a41b9a4f4364a9120e278)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:01.337303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e49dc6-392a-4b19-b182-36ba49285c20 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:01.338367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:01.343696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:01.344776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.345258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.345634 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] PATH_INFO: `/v3/projects/a064b94b61944cec9faaad2648d247ff` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.347525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a064b94b61944cec9faaad2648d247ff)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:01.349926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:01.393877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:01.414341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e8587fab7a41b9a4f4364a9120e278', 'request_id': 'req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.428672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e8587fab7a41b9a4f4364a9120e278', 'request_id': 'req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.449953 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e8587fab7a41b9a4f4364a9120e278', 'request_id': 'req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.454369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a064b94b61944cec9faaad2648d247ff', 'request_id': 'req-7662922d-ec50-4d86-9be0-3b1784d29973'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.467828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a064b94b61944cec9faaad2648d247ff', 'request_id': 'req-7662922d-ec50-4d86-9be0-3b1784d29973'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.469446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] The token cache is being invalidate because project c3e8587fab7a41b9a4f4364a9120e278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:01.469963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.472196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b374a1fa-d8c9-4bcf-b8bb-32d865474f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.475823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1065/2143] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:01 2022] DELETE /identity/v3/projects/c3e8587fab7a41b9a4f4364a9120e278 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:01.479166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a064b94b61944cec9faaad2648d247ff', 'request_id': 'req-7662922d-ec50-4d86-9be0-3b1784d29973'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.482424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476b00-7546-45d9-8a06-5ca9dd3ad182 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.482710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476b00-7546-45d9-8a06-5ca9dd3ad182 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.483088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476b00-7546-45d9-8a06-5ca9dd3ad182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.511462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] The token cache is being invalidate because project a064b94b61944cec9faaad2648d247ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:01.511877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.513255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-00476b00-7546-45d9-8a06-5ca9dd3ad182 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:01.514671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-00476b00-7546-45d9-8a06-5ca9dd3ad182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:01.520742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.523487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1079/2144] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:01 2022] DELETE /identity/v3/projects/a064b94b61944cec9faaad2648d247ff => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:40:01.527628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:01.529087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:01.615127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1066/2145] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:01.624980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c7b20-b48e-4ce2-af7e-a9b84aae557e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.625265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c7b20-b48e-4ce2-af7e-a9b84aae557e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.625545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c7b20-b48e-4ce2-af7e-a9b84aae557e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.629253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662922d-ec50-4d86-9be0-3b1784d29973 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:01.638264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:01.639226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.639584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.639976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] PATH_INFO: `/v3/users/33002b8870a348dc92b6c34c0a6ff6d7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.641427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=33002b8870a348dc92b6c34c0a6ff6d7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:01.643744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:01.655374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d05c7b20-b48e-4ce2-af7e-a9b84aae557e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:01.656597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d05c7b20-b48e-4ce2-af7e-a9b84aae557e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:01.665432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1067/2146] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:01.670148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c7b20-b48e-4ce2-af7e-a9b84aae557e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:01.681211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c7b20-b48e-4ce2-af7e-a9b84aae557e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:01.686536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:01.687215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.687493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.687766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.689061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:01.689375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:01.691519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:01.694796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:01.701667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33002b8870a348dc92b6c34c0a6ff6d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.705776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1068/2147] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:40:01.710282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:01.715686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33002b8870a348dc92b6c34c0a6ff6d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.722027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01605b8-cd77-48ca-a503-8757e2e1bc00 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:01.724777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33002b8870a348dc92b6c34c0a6ff6d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.725944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33002b8870a348dc92b6c34c0a6ff6d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.731051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:01.731721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.732011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.732292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.733006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:01.734965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:01.742114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33002b8870a348dc92b6c34c0a6ff6d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:01.753294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1080/2148] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:01 2022] DELETE /identity/v3/users/33002b8870a348dc92b6c34c0a6ff6d7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:40:01.771952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1069/2149] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:01 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:01.777914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:01.779450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:01.833868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:01.881950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd3c86-2c88-4495-8cc2-219a14f671fd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:01.946778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:01.947536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:01.947825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:01.948102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:01.949942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:01.951936 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:01.986029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1081/2150] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:01 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:01.986847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.001789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c4a12-8615-4776-9599-064cc751fe5e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.013936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.014649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.015007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.015399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.016137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.022165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.022565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.025829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:02.038817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1070/2151] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:02.042778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.044356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693e877-ceec-4c11-a036-689d3b6a6f4a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.049133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.050053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.050337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.050610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] PATH_INFO: `/v3/projects/b08d0e5a9dd947a1ad5855b13ae89996` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.052060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08d0e5a9dd947a1ad5855b13ae89996)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.054172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.066087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26dba2-0be3-4728-a5dc-73fce483856e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.078477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.079363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.079880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.080257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.081110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.085097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.113955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1071/2152] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:02.116235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.128215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08d0e5a9dd947a1ad5855b13ae89996', 'request_id': 'req-038f23fb-c6a8-4a15-b2be-ab06164b7849'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:02.129808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b97d47-01fd-434c-bcb2-00231134373b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.144700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08d0e5a9dd947a1ad5855b13ae89996', 'request_id': 'req-038f23fb-c6a8-4a15-b2be-ab06164b7849'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:02.154358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08d0e5a9dd947a1ad5855b13ae89996', 'request_id': 'req-038f23fb-c6a8-4a15-b2be-ab06164b7849'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:02.172296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] The token cache is being invalidate because project b08d0e5a9dd947a1ad5855b13ae89996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:02.172593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:02.173275 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-038f23fb-c6a8-4a15-b2be-ab06164b7849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:02.175172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1082/2153] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:02 2022] DELETE /identity/v3/projects/b08d0e5a9dd947a1ad5855b13ae89996 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:40:02.182612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda43b76-359d-443c-b646-e988cc06a36b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.183412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda43b76-359d-443c-b646-e988cc06a36b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.183691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda43b76-359d-443c-b646-e988cc06a36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.211134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-eda43b76-359d-443c-b646-e988cc06a36b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:02.212843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f13f-4d6c-432b-bc2c-cad7616571d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.213899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f13f-4d6c-432b-bc2c-cad7616571d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.214272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f13f-4d6c-432b-bc2c-cad7616571d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.214655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f13f-4d6c-432b-bc2c-cad7616571d1 admin admin] PATH_INFO: `/v3/projects/240538f0531e4d119946702e8a96a720/users/2260752f4ad948659ff46b6d06698141/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.219436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-eda43b76-359d-443c-b646-e988cc06a36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:02.233292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f13f-4d6c-432b-bc2c-cad7616571d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240538f0531e4d119946702e8a96a720, user_id=2260752f4ad948659ff46b6d06698141, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.235991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1083/2154] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:02.237883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f13f-4d6c-432b-bc2c-cad7616571d1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.244244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac0bb7-5a84-42e3-a01d-fde47479c64c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.244798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac0bb7-5a84-42e3-a01d-fde47479c64c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.245778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac0bb7-5a84-42e3-a01d-fde47479c64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.259623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1072/2155] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:02 2022] PUT /identity/v3/projects/240538f0531e4d119946702e8a96a720/users/2260752f4ad948659ff46b6d06698141/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:02.267579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f1ac0bb7-5a84-42e3-a01d-fde47479c64c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:02.269166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f1ac0bb7-5a84-42e3-a01d-fde47479c64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:02.271847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96651-d167-4409-808c-f38da96b2205 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.272235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96651-d167-4409-808c-f38da96b2205 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.272591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96651-d167-4409-808c-f38da96b2205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.302359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5bf96651-d167-4409-808c-f38da96b2205 None None] MFA Rules not processed for user `2260752f4ad948659ff46b6d06698141`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:02.303663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5bf96651-d167-4409-808c-f38da96b2205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:02.334427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:02.362258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1084/2156] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:02.363402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac0bb7-5a84-42e3-a01d-fde47479c64c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.375255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac0bb7-5a84-42e3-a01d-fde47479c64c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.381619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.382994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.383358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.383709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.384533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:02.384921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.386983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.396047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1073/2157] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:02.403464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1085/2158] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:02.407878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96651-d167-4409-808c-f38da96b2205 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.414653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.418056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:02.420901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96651-d167-4409-808c-f38da96b2205 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.428224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.428916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.429254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.429538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.430448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.433143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.461402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:02.470571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1074/2159] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:02 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:02.474748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.516226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c84f7f-e621-4183-88fa-d273c223d730 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.576677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc60d9-0330-476a-b790-1c8246e51282 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.583256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.583948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.584237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.584519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.585366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.585815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f3b6ba74fb49a280735bb8c867d26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.586626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.587007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.587353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.587464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.626060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1075/2160] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:02 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:02.633875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.641638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd352cf-c73a-4d2f-be7d-2e05d4681404 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.649527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.650391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.650799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.651852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.652750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.654641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.668683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1076/2161] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:02.683709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.683954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.685491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.699132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1086/2162] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:40:02.703885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8cced-d4cd-4c73-9d27-dd7e24b29b58 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.716419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b3cdc-c3fb-47f9-8188-6a08bf6922ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.717199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3cdc-c3fb-47f9-8188-6a08bf6922ba admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.717634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3cdc-c3fb-47f9-8188-6a08bf6922ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.718100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b3cdc-c3fb-47f9-8188-6a08bf6922ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.719087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3cdc-c3fb-47f9-8188-6a08bf6922ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.721128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b3cdc-c3fb-47f9-8188-6a08bf6922ba admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.737912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1077/2163] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:02.740449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.754358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f0cc4-e688-4dee-9a85-d29bf2c1c043 service glance] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:02.759526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:02.760508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.761404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.761773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] PATH_INFO: `/v3/projects/ad6ae5a59ff7449e9750fcf7632dcd0b/users/8906cee1071b4a1c8da0ccc05fbf4893/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.782175 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6ae5a59ff7449e9750fcf7632dcd0b, user_id=8906cee1071b4a1c8da0ccc05fbf4893, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:02.784791 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:02.818313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1087/2164] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:02 2022] PUT /identity/v3/projects/ad6ae5a59ff7449e9750fcf7632dcd0b/users/8906cee1071b4a1c8da0ccc05fbf4893/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:02.838894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:02.838894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:02.838894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:02.871573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] MFA Rules not processed for user `8906cee1071b4a1c8da0ccc05fbf4893`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:02.872723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:02.891327 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:02.893025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:02.931075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:03.021659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba8aa3-32f9-4da9-b036-ef7b213dbda1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:03.026432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:03.027147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:03.027512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:03.027795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:03.030350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:03.043594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1078/2165] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:03.106831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:03.108362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:03.118806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1088/2166] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:40:03.352734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:03.353674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:03.368370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b8761-0d0f-46fe-a9c6-8f1d58b74fbc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:03.376756 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff0e49-8a8e-492e-9428-b3645842230b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:03.377038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0e49-8a8e-492e-9428-b3645842230b service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:03.377673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0e49-8a8e-492e-9428-b3645842230b service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:03.378307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0e49-8a8e-492e-9428-b3645842230b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:03.390928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff0e49-8a8e-492e-9428-b3645842230b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:03.392946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff0e49-8a8e-492e-9428-b3645842230b service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:03.406343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1079/2167] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:03.430507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:03.564371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed088b2-4921-4c12-a508-781e15f23ef7 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:03.570584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:03.571421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:03.571669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:03.571953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] PATH_INFO: `/v3/users/8906cee1071b4a1c8da0ccc05fbf4893` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:03.573269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8906cee1071b4a1c8da0ccc05fbf4893)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:03.575616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:03.647370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8906cee1071b4a1c8da0ccc05fbf4893'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:03.659959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8906cee1071b4a1c8da0ccc05fbf4893'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:03.673174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8906cee1071b4a1c8da0ccc05fbf4893'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:03.673953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8906cee1071b4a1c8da0ccc05fbf4893'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:03.684199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:03.690234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8906cee1071b4a1c8da0ccc05fbf4893'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:03.697946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1089/2168] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:03 2022] DELETE /identity/v3/users/8906cee1071b4a1c8da0ccc05fbf4893 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:03.702711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:03.703201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:03.704002 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:03.723449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:03.724818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:03.770691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:03.796325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1080/2169] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:03.806124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:03.807622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:03.918717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e91977-7ccb-4c95-81c9-1c2f71c206fa admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:03.925620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:03.926529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:03.926866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:03.927290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:03.941297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:03.943157 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:03.959970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1090/2170] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:04.302650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:04.303987 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:04.417440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a9ee6-59bc-466a-bf25-c46d62fbee51 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:04.423547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca70b9-af7d-4305-ae85-26d60ff5d40d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:04.424349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca70b9-af7d-4305-ae85-26d60ff5d40d service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:04.424833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca70b9-af7d-4305-ae85-26d60ff5d40d service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:04.425112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca70b9-af7d-4305-ae85-26d60ff5d40d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:04.429573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-96ca70b9-af7d-4305-ae85-26d60ff5d40d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:04.539135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:04.541474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:04.572967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54589fe-12ea-4bd2-9846-f2691f2b5abe service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:04.583869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:04.585130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:04.585617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:04.586304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] PATH_INFO: `/v3/projects/ad6ae5a59ff7449e9750fcf7632dcd0b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:04.588025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6ae5a59ff7449e9750fcf7632dcd0b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:04.592653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:04.602081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca70b9-af7d-4305-ae85-26d60ff5d40d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:04.606812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca70b9-af7d-4305-ae85-26d60ff5d40d service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:04.655241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1081/2171] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:04.723676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6ae5a59ff7449e9750fcf7632dcd0b', 'request_id': 'req-c7761cd2-051e-434f-bfba-38717b639421'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:04.748676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6ae5a59ff7449e9750fcf7632dcd0b', 'request_id': 'req-c7761cd2-051e-434f-bfba-38717b639421'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:04.774627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6ae5a59ff7449e9750fcf7632dcd0b', 'request_id': 'req-c7761cd2-051e-434f-bfba-38717b639421'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:04.799840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] The token cache is being invalidate because project ad6ae5a59ff7449e9750fcf7632dcd0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:04.800139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:04.802888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c7761cd2-051e-434f-bfba-38717b639421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:04.810055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1091/2172] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:04 2022] DELETE /identity/v3/projects/ad6ae5a59ff7449e9750fcf7632dcd0b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:04.821447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbcdd0-3f97-4092-a9e6-01145b4c7198 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:04.822502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbcdd0-3f97-4092-a9e6-01145b4c7198 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:04.822866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbcdd0-3f97-4092-a9e6-01145b4c7198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:04.850697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c4dbcdd0-3f97-4092-a9e6-01145b4c7198 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:04.851945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c4dbcdd0-3f97-4092-a9e6-01145b4c7198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:04.923169 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1082/2173] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:04.931179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:04.931607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:04.931767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:04.951695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:04.952911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:04.962079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1092/2174] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:04.966142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbcdd0-3f97-4092-a9e6-01145b4c7198 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:04.977013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbcdd0-3f97-4092-a9e6-01145b4c7198 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:04.982478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:04.983300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:04.983675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:04.984006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:04.984834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:04.985206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:04.986470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:04.987033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:04.990888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:05.004865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1e60f-bae5-47db-b70c-f92f3e5e8d51 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.005769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1083/2175] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:05.010082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.010353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.010983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.011263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.011535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] PATH_INFO: `/v3/users/6e2a2c6831ba4d13aeb6e550f8eff9a6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.012730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2a2c6831ba4d13aeb6e550f8eff9a6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.014676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.020281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fa645-ed6b-42d7-8400-e9269ee130db admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.024928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.025579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.025886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.027188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.027894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.029914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.052567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:05.060866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1084/2176] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:05.064624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.067752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2a2c6831ba4d13aeb6e550f8eff9a6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.078292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2a2c6831ba4d13aeb6e550f8eff9a6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.087104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2a2c6831ba4d13aeb6e550f8eff9a6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.087990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2a2c6831ba4d13aeb6e550f8eff9a6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.090127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:05.102847 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2a2c6831ba4d13aeb6e550f8eff9a6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.109811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1093/2177] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:04 2022] DELETE /identity/v3/users/6e2a2c6831ba4d13aeb6e550f8eff9a6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:05.129929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cd9de-a751-4d93-9490-01dfbdf3359f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.191155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.193202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.193202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.193202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.193202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.195759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.223341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1085/2178] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:05.224816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.226527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:05.242851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0926a-2d1e-4129-b013-2e241f0c3777 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.249037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.249768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.250097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.250957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.251712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.253635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.270605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1094/2179] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:05.274657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.286964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff4db8-e4ef-47c2-81a3-b80c0171c4b2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.292871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.293640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.293965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.294245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.294979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.296696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.300793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.302746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:05.316965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1086/2180] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:05.321652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.328395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58056cd-2ad9-4d75-a902-a3978d1c65d9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.333452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1d979-00c3-4767-b600-81a0c9d5f901 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.334464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.335336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.335620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.335952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] PATH_INFO: `/v3/projects/8a16c8fcc2f046fd9b1309ea822eb5b5` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.337420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a16c8fcc2f046fd9b1309ea822eb5b5)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.340102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.340778 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647f23-b465-4fb1-97b3-c48054b676f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.341822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647f23-b465-4fb1-97b3-c48054b676f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.342218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647f23-b465-4fb1-97b3-c48054b676f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.342559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647f23-b465-4fb1-97b3-c48054b676f3 admin admin] PATH_INFO: `/v3/projects/9bb99b7886f146ee99a9b9c348a61db5/users/3b1d2149c3f54feb96c12e41db48c295/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.359326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647f23-b465-4fb1-97b3-c48054b676f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb99b7886f146ee99a9b9c348a61db5, user_id=3b1d2149c3f54feb96c12e41db48c295, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.362053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647f23-b465-4fb1-97b3-c48054b676f3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.375488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1087/2181] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:05 2022] PUT /identity/v3/projects/9bb99b7886f146ee99a9b9c348a61db5/users/3b1d2149c3f54feb96c12e41db48c295/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:05.386922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.387475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.388231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.394115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a16c8fcc2f046fd9b1309ea822eb5b5', 'request_id': 'req-e55a627f-7b3d-4640-a7e3-caa009c4629c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.405115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a16c8fcc2f046fd9b1309ea822eb5b5', 'request_id': 'req-e55a627f-7b3d-4640-a7e3-caa009c4629c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.412223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a16c8fcc2f046fd9b1309ea822eb5b5', 'request_id': 'req-e55a627f-7b3d-4640-a7e3-caa009c4629c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.420450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] MFA Rules not processed for user `3b1d2149c3f54feb96c12e41db48c295`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:05.421711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:05.427211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] The token cache is being invalidate because project 8a16c8fcc2f046fd9b1309ea822eb5b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:05.427610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.428612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55a627f-7b3d-4640-a7e3-caa009c4629c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:05.434672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1095/2182] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:05 2022] DELETE /identity/v3/projects/8a16c8fcc2f046fd9b1309ea822eb5b5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:05.437357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60100ea-f5f8-4965-b311-9c53b7121e0c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.437357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60100ea-f5f8-4965-b311-9c53b7121e0c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.437357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60100ea-f5f8-4965-b311-9c53b7121e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.457359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c60100ea-f5f8-4965-b311-9c53b7121e0c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:05.458620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c60100ea-f5f8-4965-b311-9c53b7121e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:05.481125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:05.499195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1088/2183] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:05.510883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.512177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:05.520901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1096/2184] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:05.529666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311206-7d8b-4d43-8134-019946b73a6d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.530195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311206-7d8b-4d43-8134-019946b73a6d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.530407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95311206-7d8b-4d43-8134-019946b73a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.552645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-95311206-7d8b-4d43-8134-019946b73a6d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:05.553878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-95311206-7d8b-4d43-8134-019946b73a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:05.561948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1097/2185] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:05.566050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95311206-7d8b-4d43-8134-019946b73a6d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.575445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95311206-7d8b-4d43-8134-019946b73a6d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.581309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5123b-c340-4e8b-8d76-cf053cbcc20f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.581444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.582017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.582306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.582579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.583312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:05.583620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.585468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.585962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.586197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.586485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.586839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.598023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.599536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1098/2186] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:40:05.600042 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.605450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.612734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1089/2187] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:05.616625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ea220-2394-4418-b754-87599206d355 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.627729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.628253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.635291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.635291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.635291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.635291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.662276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1099/2188] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:05.666361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.679392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfbdc5c-6305-4c6f-8396-23c3e290a321 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.748994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.749667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.749993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.750272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.750980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.752973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.776611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1090/2189] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:05.778017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.788754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3f55f-1371-45e3-9efa-e30abd43ce69 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.795802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.796586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.797024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.797521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.798332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.800193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.812330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1100/2190] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:05.821588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.837795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77044a31-3232-4f2f-a6cd-d956d56eb5ab admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.846664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6ef99-96d0-4b3f-a9a6-e36a6b4094f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.846664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ef99-96d0-4b3f-a9a6-e36a6b4094f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.846664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ef99-96d0-4b3f-a9a6-e36a6b4094f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.846664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ef99-96d0-4b3f-a9a6-e36a6b4094f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.846976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6ef99-96d0-4b3f-a9a6-e36a6b4094f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.848225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6ef99-96d0-4b3f-a9a6-e36a6b4094f7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.864618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1091/2191] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:05.874237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:05.882268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:05.890403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3bdf3-c2ff-454f-ad9d-35b73ad48f30 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:05.897555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:05.899163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.899581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.899972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] PATH_INFO: `/v3/projects/a0f2b3c7fb15462d8a230d488534fe18/users/9b2a8aa8f4724e48a41afce944657ba5/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:05.925078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f2b3c7fb15462d8a230d488534fe18, user_id=9b2a8aa8f4724e48a41afce944657ba5, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:05.929024 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:05.969982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1101/2192] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:05 2022] PUT /identity/v3/projects/a0f2b3c7fb15462d8a230d488534fe18/users/9b2a8aa8f4724e48a41afce944657ba5/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:05.972457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb3b12-ff21-407c-bf2d-1037b0cae5d4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:05.972777 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb3b12-ff21-407c-bf2d-1037b0cae5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:05.973262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb3b12-ff21-407c-bf2d-1037b0cae5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:06.014921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-becb3b12-ff21-407c-bf2d-1037b0cae5d4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:06.016235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-becb3b12-ff21-407c-bf2d-1037b0cae5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:06.104020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1092/2193] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:06.114590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:06.116424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:06.196607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3c3f0-2ff1-4ce7-a1b6-870582b2b546 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:06.200719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:06.201522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:06.201801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:06.202121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:06.214253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:06.215688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:06.226197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1102/2194] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:10.681504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:10.682034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:10.682124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:10.708272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:10.713174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] MFA Rules not processed for user `9b2a8aa8f4724e48a41afce944657ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:10.716216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:10.797749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1093/2195] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:10.813357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:10.814939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:10.898468 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a4bf5-f8b6-4d3c-a059-9e532a50e9dd service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:10.905111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bf4c5-b6cc-4616-8092-ffa2bb220c83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:10.905779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bf4c5-b6cc-4616-8092-ffa2bb220c83 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:10.906119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bf4c5-b6cc-4616-8092-ffa2bb220c83 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:10.906542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bf4c5-b6cc-4616-8092-ffa2bb220c83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:10.915577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bf4c5-b6cc-4616-8092-ffa2bb220c83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:10.917277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bf4c5-b6cc-4616-8092-ffa2bb220c83 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:10.927294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1103/2196] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:12.123486 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:12.125556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:12.182909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bd287-c996-4c0e-b9c3-185476b5e673 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:12.194948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:12.196654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:12.198441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:12.199569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] PATH_INFO: `/v3/users/c8c4d7e6692142908031e4b05042707a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:12.202846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c4d7e6692142908031e4b05042707a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:12.208212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:12.336653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c4d7e6692142908031e4b05042707a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:12.347122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c4d7e6692142908031e4b05042707a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:12.359157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c4d7e6692142908031e4b05042707a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:12.362391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c4d7e6692142908031e4b05042707a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:12.366614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:12.367161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:12.367161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:12.380059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-193ffd8a-e0d5-4990-ae07-62c1758a5a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c4d7e6692142908031e4b05042707a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:12.389889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1094/2197] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:12 2022] DELETE /identity/v3/users/c8c4d7e6692142908031e4b05042707a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:40:12.391361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] MFA Rules not processed for user `9b2a8aa8f4724e48a41afce944657ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:12.393232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:12.398206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:12.399745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:12.400253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:12.421501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:12.422805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:12.470864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1104/2198] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:12.502490 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1095/2199] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:12.508876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:12.510551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:12.535776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:12.598487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631ba68-718a-4a21-b0be-7add275080e7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:12.610428 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:12.613759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:12.613759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:12.613759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:12.634529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:12.634529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:12.643150 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1105/2200] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:12.941346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:12.943669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:12.962447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb8ecf-05b5-461e-b1e9-0706e7aeb2d4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:12.968119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:12.969019 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:12.969303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:12.969581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] PATH_INFO: `/v3/projects/d41282deaa8e415893d84f60831e737c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:12.971125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41282deaa8e415893d84f60831e737c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:12.972959 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:12.993013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:13.037723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41282deaa8e415893d84f60831e737c', 'request_id': 'req-e7b9c88f-8bf2-4673-85da-ea4f586730f1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.047457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41282deaa8e415893d84f60831e737c', 'request_id': 'req-e7b9c88f-8bf2-4673-85da-ea4f586730f1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.053823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41282deaa8e415893d84f60831e737c', 'request_id': 'req-e7b9c88f-8bf2-4673-85da-ea4f586730f1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.069400 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] The token cache is being invalidate because project d41282deaa8e415893d84f60831e737c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:13.069704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.070417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.072325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1096/2201] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:12 2022] DELETE /identity/v3/projects/d41282deaa8e415893d84f60831e737c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:13.076106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:13.086179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab172c9-540c-4a5a-8e98-ddf45f1de2a5 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:13.198826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:13.199726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:13.200010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:13.200326 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] PATH_INFO: `/v3/users/a2b1c8fbee524103a9c91023c774f340` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:13.201943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b1c8fbee524103a9c91023c774f340)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:13.204266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:13.287511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b1c8fbee524103a9c91023c774f340'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.298168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b1c8fbee524103a9c91023c774f340'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.305924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b1c8fbee524103a9c91023c774f340'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.306600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b1c8fbee524103a9c91023c774f340'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.324236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b1c8fbee524103a9c91023c774f340'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:13.333393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1106/2202] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:13 2022] DELETE /identity/v3/users/a2b1c8fbee524103a9c91023c774f340 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:13.833159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:13.835472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:13.871168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9c88f-8bf2-4673-85da-ea4f586730f1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:13.966618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:13.967509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:13.967971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:13.968303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] PATH_INFO: `/v3/projects/7283130273b245e782269b94cf6638a1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:13.969604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7283130273b245e782269b94cf6638a1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:13.971738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:14.032276 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7283130273b245e782269b94cf6638a1', 'request_id': 'req-095b7d12-78cc-4757-b706-3410b7513bf3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.043354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7283130273b245e782269b94cf6638a1', 'request_id': 'req-095b7d12-78cc-4757-b706-3410b7513bf3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.059234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7283130273b245e782269b94cf6638a1', 'request_id': 'req-095b7d12-78cc-4757-b706-3410b7513bf3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.072856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] The token cache is being invalidate because project 7283130273b245e782269b94cf6638a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:14.073145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.073819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.075763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1097/2203] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:13 2022] DELETE /identity/v3/projects/7283130273b245e782269b94cf6638a1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:14.080089 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:14.081725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:14.095343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9db0-4fe6-43ae-85c9-f998c4587768 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:14.135862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:14.180021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:14.180910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:14.181192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:14.181994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] PATH_INFO: `/v3/users/6c991f70e7b44a129d9c163e8302ab69` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:14.183559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c991f70e7b44a129d9c163e8302ab69)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:14.185498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:14.268270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c991f70e7b44a129d9c163e8302ab69'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.281921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c991f70e7b44a129d9c163e8302ab69'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.293786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c991f70e7b44a129d9c163e8302ab69'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.294565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c991f70e7b44a129d9c163e8302ab69'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.310269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c991f70e7b44a129d9c163e8302ab69'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.324426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1107/2204] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:14 2022] DELETE /identity/v3/users/6c991f70e7b44a129d9c163e8302ab69 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:14.641498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:14.643217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:14.664503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b7d12-78cc-4757-b706-3410b7513bf3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:14.693307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:14.729658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:14.730664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:14.730946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:14.731221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] PATH_INFO: `/v3/projects/0c50f01fb0cf4a948a4923347834013d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:14.732497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c50f01fb0cf4a948a4923347834013d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:14.734362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:14.805295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c50f01fb0cf4a948a4923347834013d', 'request_id': 'req-e49cbf56-bcc7-43fd-8709-4100960542f1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.817083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c50f01fb0cf4a948a4923347834013d', 'request_id': 'req-e49cbf56-bcc7-43fd-8709-4100960542f1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.825611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c50f01fb0cf4a948a4923347834013d', 'request_id': 'req-e49cbf56-bcc7-43fd-8709-4100960542f1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.840839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] The token cache is being invalidate because project 0c50f01fb0cf4a948a4923347834013d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:14.844170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.845224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e49cbf56-bcc7-43fd-8709-4100960542f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:14.861867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1098/2205] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:14 2022] DELETE /identity/v3/projects/0c50f01fb0cf4a948a4923347834013d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:24.319206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=38,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:24.834192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:31.766576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:31.768832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:31.797220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d333a-ae75-45f3-9aaa-e3d6c9fa6b28 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:31.862872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:31.863781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:31.864066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:31.864348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] PATH_INFO: `/v3/users/3b1d2149c3f54feb96c12e41db48c295` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:31.865593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b1d2149c3f54feb96c12e41db48c295)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:31.867612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:31.934556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b1d2149c3f54feb96c12e41db48c295'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:31.945163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b1d2149c3f54feb96c12e41db48c295'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:31.954315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b1d2149c3f54feb96c12e41db48c295'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:31.955062 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b1d2149c3f54feb96c12e41db48c295'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:31.969669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b1d2149c3f54feb96c12e41db48c295'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:31.976346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1108/2206] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:31 2022] DELETE /identity/v3/users/3b1d2149c3f54feb96c12e41db48c295 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:40:31.980628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:31.981114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:31.981195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.001106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:32.002376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:32.089029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1099/2207] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:32.099403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:32.100951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:32.181820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e228c-6ce2-4668-86d0-2b47dbbe5621 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:32.185485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4037d-52ab-47b8-bc2a-ee6c90d97186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:32.186190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4037d-52ab-47b8-bc2a-ee6c90d97186 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:32.186478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4037d-52ab-47b8-bc2a-ee6c90d97186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:32.186754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4037d-52ab-47b8-bc2a-ee6c90d97186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.196828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4037d-52ab-47b8-bc2a-ee6c90d97186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:32.198429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4037d-52ab-47b8-bc2a-ee6c90d97186 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:32.211757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1109/2208] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:32.482595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:32.485573 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:32.507489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9a7f9-700d-4a51-89af-5525a9f527fe None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:32.512719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:32.513592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:32.513996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:32.514209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] PATH_INFO: `/v3/projects/9bb99b7886f146ee99a9b9c348a61db5` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.516290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb99b7886f146ee99a9b9c348a61db5)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:32.518115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:32.568345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb99b7886f146ee99a9b9c348a61db5', 'request_id': 'req-a3a86511-4b75-4472-abbf-15d5f1f46370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:32.576351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb99b7886f146ee99a9b9c348a61db5', 'request_id': 'req-a3a86511-4b75-4472-abbf-15d5f1f46370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:32.582866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb99b7886f146ee99a9b9c348a61db5', 'request_id': 'req-a3a86511-4b75-4472-abbf-15d5f1f46370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:32.595228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] The token cache is being invalidate because project 9bb99b7886f146ee99a9b9c348a61db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:32.595524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:32.596202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3a86511-4b75-4472-abbf-15d5f1f46370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:32.598037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1100/2209] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:32 2022] DELETE /identity/v3/projects/9bb99b7886f146ee99a9b9c348a61db5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:32.602799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f58f7-04e4-4aa3-a2b0-225b7e61e167 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:32.603159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f58f7-04e4-4aa3-a2b0-225b7e61e167 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:32.603534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f58f7-04e4-4aa3-a2b0-225b7e61e167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.622133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f83f58f7-04e4-4aa3-a2b0-225b7e61e167 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:32.623586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f83f58f7-04e4-4aa3-a2b0-225b7e61e167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:32.645410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:32.697761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1110/2210] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:32.706804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f49ac-4878-4771-bfcd-2888506027c5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:32.707099 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f49ac-4878-4771-bfcd-2888506027c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:32.707387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f49ac-4878-4771-bfcd-2888506027c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.728695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-987f49ac-4878-4771-bfcd-2888506027c5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:32.730410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-987f49ac-4878-4771-bfcd-2888506027c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:32.742109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1101/2211] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:32.746217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f58f7-04e4-4aa3-a2b0-225b7e61e167 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:32.756388 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f58f7-04e4-4aa3-a2b0-225b7e61e167 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:32.761086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:32.761764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:32.762084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:32.762366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.763230 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:32.763557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:32.765878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:32.778777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1111/2212] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:32.784087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f49ac-4878-4771-bfcd-2888506027c5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:32.794989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f49ac-4878-4771-bfcd-2888506027c5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:32.800501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:32.801204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:32.801491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:32.801771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.802554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:32.804506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:32.837118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1102/2213] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:32 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:32.838666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:32.849423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d527418-1baa-4de5-bbc3-ab36a775df1d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:32.945633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:32.946443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:32.946734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:32.947616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:32.947793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:32.949861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:32.982330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1112/2214] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:32 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:32.984220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:32.997158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5762b0a-23c1-4f46-b489-c2e0b188ba49 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:33.003092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:33.003791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:33.004079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:33.004363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:33.005112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:33.008003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:33.025170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1103/2215] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:33.030515 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:33.047403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93b83e-a34e-4244-b142-c6cf2eefb97f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:33.054360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3580f-85ee-4fb8-8efb-4b38424522fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:33.055096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3580f-85ee-4fb8-8efb-4b38424522fb admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:33.055571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3580f-85ee-4fb8-8efb-4b38424522fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:33.055880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3580f-85ee-4fb8-8efb-4b38424522fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:33.056794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3580f-85ee-4fb8-8efb-4b38424522fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:33.058703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3580f-85ee-4fb8-8efb-4b38424522fb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:33.079304 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1113/2216] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:40:33.081167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:33.092913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a6385-01cc-4f13-bc71-541abd238df6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:33.098670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:33.099825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:33.100185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:33.100545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] PATH_INFO: `/v3/projects/5c8c935ef81f487ea6cac72f452bdc5d/users/ea8c61c001fe4eafa66c35bc8430f235/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:33.128466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c8c935ef81f487ea6cac72f452bdc5d, user_id=ea8c61c001fe4eafa66c35bc8430f235, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:33.131438 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:33.161289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1104/2217] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:33 2022] PUT /identity/v3/projects/5c8c935ef81f487ea6cac72f452bdc5d/users/ea8c61c001fe4eafa66c35bc8430f235/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:33.166740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0dee0-700c-4c3f-b4fe-b823b70226ac None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:33.167043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0dee0-700c-4c3f-b4fe-b823b70226ac None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:33.167320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0dee0-700c-4c3f-b4fe-b823b70226ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:33.190824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4de0dee0-700c-4c3f-b4fe-b823b70226ac None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:33.192129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4de0dee0-700c-4c3f-b4fe-b823b70226ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:33.292966 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1114/2218] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:33.303792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:33.306371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:33.338250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:33.385052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73ff86-d7f6-4bc3-b394-f2b16cc4a472 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:33.389640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:33.390520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:33.390971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:33.391375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:33.404579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:33.407371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:33.418488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1105/2219] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:34.259558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc7e8-3143-48a7-881f-16576c39a127 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:34.260001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc7e8-3143-48a7-881f-16576c39a127 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:34.260231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dc7e8-3143-48a7-881f-16576c39a127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:34.302807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-291dc7e8-3143-48a7-881f-16576c39a127 None None] MFA Rules not processed for user `ea8c61c001fe4eafa66c35bc8430f235`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:34.304376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-291dc7e8-3143-48a7-881f-16576c39a127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:34.328543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:34.398609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1115/2220] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:34.410517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:34.412542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:34.470848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f759e1-db58-4ffc-ad53-2a1c654d260c service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:34.474366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:34.475017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:34.475301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:34.475576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:34.483982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:34.485401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:34.494213 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1106/2221] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:35.888490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cc574-22f0-429a-bb55-44fe89eccb8b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:35.889037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cc574-22f0-429a-bb55-44fe89eccb8b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:35.889158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422cc574-22f0-429a-bb55-44fe89eccb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:35.909110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-422cc574-22f0-429a-bb55-44fe89eccb8b None None] MFA Rules not processed for user `ea8c61c001fe4eafa66c35bc8430f235`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:35.910359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-422cc574-22f0-429a-bb55-44fe89eccb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:35.917708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1116/2222] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:40.338219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:40.339625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:40.354015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e3729-8c01-48b1-a4f3-26518ac9d45d service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:40.358411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:40.359269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:40.359552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:40.359826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] PATH_INFO: `/v3/users/3f773aab53304103a72b27226cc1b729` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:40.361680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f773aab53304103a72b27226cc1b729)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:40.363627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:40.409061 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f773aab53304103a72b27226cc1b729'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:40.415917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f773aab53304103a72b27226cc1b729'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:40.423812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f773aab53304103a72b27226cc1b729'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:40.424736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f773aab53304103a72b27226cc1b729'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:40.439016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f773aab53304103a72b27226cc1b729'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:40.445089 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1107/2223] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:40 2022] DELETE /identity/v3/users/3f773aab53304103a72b27226cc1b729 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:40.449654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:40.450127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:40.450245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:40.467182 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:40.468413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:40.526752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1117/2224] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:40.537003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:40.538432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:40.579920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:40.597223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8bc47d-8019-4f76-9204-da42f7927949 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:40.600865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bb789-bfc9-4adf-a887-641516bdceb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:40.601516 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bb789-bfc9-4adf-a887-641516bdceb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:40.601804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bb789-bfc9-4adf-a887-641516bdceb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:40.602116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bb789-bfc9-4adf-a887-641516bdceb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:40.611962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bb789-bfc9-4adf-a887-641516bdceb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:40.613392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bb789-bfc9-4adf-a887-641516bdceb5 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:40.623937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1108/2225] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:40.930026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:40.931471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:40.943513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9144e0-40e2-4dd5-8791-75d9488622b8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:40.948223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:40.949086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:40.949369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:40.949702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] PATH_INFO: `/v3/projects/9a58e7ee487b438f8641edce5eea3d34` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:40.950950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a58e7ee487b438f8641edce5eea3d34)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:40.952865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.008528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a58e7ee487b438f8641edce5eea3d34', 'request_id': 'req-e55d744c-1852-450f-a9ee-78d43f8b345e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:41.019732 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a58e7ee487b438f8641edce5eea3d34', 'request_id': 'req-e55d744c-1852-450f-a9ee-78d43f8b345e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:41.025577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a58e7ee487b438f8641edce5eea3d34', 'request_id': 'req-e55d744c-1852-450f-a9ee-78d43f8b345e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:41.037760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] The token cache is being invalidate because project 9a58e7ee487b438f8641edce5eea3d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:41.038086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:41.038709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e55d744c-1852-450f-a9ee-78d43f8b345e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:41.040239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1118/2226] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:40 2022] DELETE /identity/v3/projects/9a58e7ee487b438f8641edce5eea3d34 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:41.045449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137d99-2e7f-4f42-961a-ab1acf269d06 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.045738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137d99-2e7f-4f42-961a-ab1acf269d06 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.046049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137d99-2e7f-4f42-961a-ab1acf269d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.065274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ac137d99-2e7f-4f42-961a-ab1acf269d06 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:41.066515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ac137d99-2e7f-4f42-961a-ab1acf269d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:41.128171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1109/2227] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:41.136110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765feb-1987-46f2-ad56-d4ed67b5ccaa None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.136400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765feb-1987-46f2-ad56-d4ed67b5ccaa None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.136679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf765feb-1987-46f2-ad56-d4ed67b5ccaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.156251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bf765feb-1987-46f2-ad56-d4ed67b5ccaa None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:41.157465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bf765feb-1987-46f2-ad56-d4ed67b5ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:41.165549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1119/2228] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:41.169787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac137d99-2e7f-4f42-961a-ab1acf269d06 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:41.179301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac137d99-2e7f-4f42-961a-ab1acf269d06 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:41.185063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:41.185737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.186048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.186325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.187338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:41.187338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:41.189512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.201623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1110/2229] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:41.206535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf765feb-1987-46f2-ad56-d4ed67b5ccaa None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:41.219483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf765feb-1987-46f2-ad56-d4ed67b5ccaa None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:41.224726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:41.225392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.225679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.226112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.226818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:41.229360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.260632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1120/2230] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:41 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:41.261561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:41.273774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ec156-03b8-4f84-b5ef-d4953fbf180b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:41.344128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:41.345568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.346085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.346471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.347181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:41.349210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.374237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1111/2231] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:41 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:41.378122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:41.391893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb47b-9a5c-4574-ae9a-6abd542c1eb0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:41.399176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:41.401112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.401473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.401870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.404193 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:41.405970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.408643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:41.424643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1121/2232] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:41.427652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:41.441563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43466ab-90da-4131-a9af-37167df779b6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:41.446750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6138ab-fdc1-4c1b-8ed9-cc0c3c4fd80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:41.447479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6138ab-fdc1-4c1b-8ed9-cc0c3c4fd80e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.447863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6138ab-fdc1-4c1b-8ed9-cc0c3c4fd80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.448202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6138ab-fdc1-4c1b-8ed9-cc0c3c4fd80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.448934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6138ab-fdc1-4c1b-8ed9-cc0c3c4fd80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:41.451000 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6138ab-fdc1-4c1b-8ed9-cc0c3c4fd80e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.464177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1112/2233] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:40:41.468452 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:41.479480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18568-594c-4a22-b7ae-e4de5cd3ee93 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:41.484227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:41.485132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.485415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.485690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] PATH_INFO: `/v3/projects/934dee1c265a4abdaaa96d455878bed9/users/d8f6fd7bc45d4a0198459bff3c0d8fd4/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.500159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=934dee1c265a4abdaaa96d455878bed9, user_id=d8f6fd7bc45d4a0198459bff3c0d8fd4, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:41.502822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.515599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1122/2234] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:41 2022] PUT /identity/v3/projects/934dee1c265a4abdaaa96d455878bed9/users/d8f6fd7bc45d4a0198459bff3c0d8fd4/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:41.527175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e32ae7-bcea-46e9-b121-14bccd6fddcd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.527466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e32ae7-bcea-46e9-b121-14bccd6fddcd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.527743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e32ae7-bcea-46e9-b121-14bccd6fddcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.555040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f6e32ae7-bcea-46e9-b121-14bccd6fddcd None None] MFA Rules not processed for user `d8f6fd7bc45d4a0198459bff3c0d8fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:41.556327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f6e32ae7-bcea-46e9-b121-14bccd6fddcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:41.589556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:41.621066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1113/2235] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:41.636050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:41.638192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:41.702348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1505f1-bf35-48bf-941b-aeb887899fae admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:41.707580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be068cf-5640-44ec-acb5-5066a831c88f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:41.708920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be068cf-5640-44ec-acb5-5066a831c88f service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:41.709055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be068cf-5640-44ec-acb5-5066a831c88f service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:41.709305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be068cf-5640-44ec-acb5-5066a831c88f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:41.721957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be068cf-5640-44ec-acb5-5066a831c88f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:41.723395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be068cf-5640-44ec-acb5-5066a831c88f service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:41.736444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1123/2236] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:51.618073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=11 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:51.736484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:55.171719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e32ae7-bcea-46e9-b121-14bccd6fddcd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:55.347955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e32ae7-bcea-46e9-b121-14bccd6fddcd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:55.353755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:55.354763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:55.355050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:55.355495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] PATH_INFO: `/v3/users/d8f6fd7bc45d4a0198459bff3c0d8fd4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:55.356905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f6fd7bc45d4a0198459bff3c0d8fd4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:55.360114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:55.414068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f6fd7bc45d4a0198459bff3c0d8fd4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:55.429754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f6fd7bc45d4a0198459bff3c0d8fd4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:55.437902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f6fd7bc45d4a0198459bff3c0d8fd4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:55.439329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f6fd7bc45d4a0198459bff3c0d8fd4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:55.452106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f6fd7bc45d4a0198459bff3c0d8fd4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:55.458928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1114/2237] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:55 2022] DELETE /identity/v3/users/d8f6fd7bc45d4a0198459bff3c0d8fd4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:55.463485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:55.463986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:55.464134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:55.514823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:55.516044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:55.578094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1124/2238] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:55.588418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:55.590011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:55.636091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:55.701470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e08f9-2221-4399-ac9b-4e97a8c7de8f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:55.705284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ea0f8-e23e-4d84-9b8e-1bdb1cfb53de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:55.706097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ea0f8-e23e-4d84-9b8e-1bdb1cfb53de service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:55.706367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ea0f8-e23e-4d84-9b8e-1bdb1cfb53de service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:55.706654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ea0f8-e23e-4d84-9b8e-1bdb1cfb53de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:55.716372 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ea0f8-e23e-4d84-9b8e-1bdb1cfb53de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:55.717792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ea0f8-e23e-4d84-9b8e-1bdb1cfb53de service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:55.727506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1115/2239] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:56.006795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.008391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:56.021064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbad210-e9f9-443e-bdc0-cf25c529d8f4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.026128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.027378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.027674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.027988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] PATH_INFO: `/v3/projects/934dee1c265a4abdaaa96d455878bed9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.029232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934dee1c265a4abdaaa96d455878bed9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.031239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.079951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934dee1c265a4abdaaa96d455878bed9', 'request_id': 'req-3c419557-c803-4c3d-afd2-a85416de1e17'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:56.087710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934dee1c265a4abdaaa96d455878bed9', 'request_id': 'req-3c419557-c803-4c3d-afd2-a85416de1e17'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:56.094536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934dee1c265a4abdaaa96d455878bed9', 'request_id': 'req-3c419557-c803-4c3d-afd2-a85416de1e17'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:56.106777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] The token cache is being invalidate because project 934dee1c265a4abdaaa96d455878bed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:56.107147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:56.107810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3c419557-c803-4c3d-afd2-a85416de1e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:56.109397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1125/2240] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:56 2022] DELETE /identity/v3/projects/934dee1c265a4abdaaa96d455878bed9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:56.116178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec8618-825f-4b2a-8baa-bba4c2bb52e9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.116527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec8618-825f-4b2a-8baa-bba4c2bb52e9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.116812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec8618-825f-4b2a-8baa-bba4c2bb52e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.157203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9bec8618-825f-4b2a-8baa-bba4c2bb52e9 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:56.159426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9bec8618-825f-4b2a-8baa-bba4c2bb52e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:56.227166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1116/2241] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:56.235034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1045-ed2b-4709-82ab-a0a902b1428a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.235331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1045-ed2b-4709-82ab-a0a902b1428a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.235762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1045-ed2b-4709-82ab-a0a902b1428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.257507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-961e1045-ed2b-4709-82ab-a0a902b1428a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:56.258825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-961e1045-ed2b-4709-82ab-a0a902b1428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:56.267850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1126/2242] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:56.271961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec8618-825f-4b2a-8baa-bba4c2bb52e9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.282512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec8618-825f-4b2a-8baa-bba4c2bb52e9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.287671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.288357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.288649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.288927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.289667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:56.290009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.291626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.303634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1117/2243] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:56.307900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1045-ed2b-4709-82ab-a0a902b1428a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.317833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1045-ed2b-4709-82ab-a0a902b1428a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.322601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.323256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.323537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.323809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.324543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.326543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.355694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1127/2244] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:56 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:56.360297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.370329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad956f91-5fc0-4241-bf60-501a967668fa admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.442835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.443526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.443809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.444087 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.444926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.447343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.469237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1118/2245] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:56 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:40:56.473548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.489065 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a822424-10c5-4190-bf22-c2d5c3630c56 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.495171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.495915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.496200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.496531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.498191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.500325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.525793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1128/2246] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:40:56.529906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.540964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3cdca-4637-4150-9247-95517c670b3e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.546164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa30e3e-4a26-48ab-afc9-8282bede72f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.546843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30e3e-4a26-48ab-afc9-8282bede72f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.547125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30e3e-4a26-48ab-afc9-8282bede72f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.547398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30e3e-4a26-48ab-afc9-8282bede72f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.548710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa30e3e-4a26-48ab-afc9-8282bede72f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.550404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa30e3e-4a26-48ab-afc9-8282bede72f2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.551806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:56.562903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1119/2247] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:56.567766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.578071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8ada7-3fc0-43e8-a990-9ce2d536824f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.582779 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.583656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.583944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.584222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] PATH_INFO: `/v3/projects/f00dee2ecea141ed9829009f1d2e0ea1/users/27d4e54f75754129bd81de7a6d5a2a05/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.598872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00dee2ecea141ed9829009f1d2e0ea1, user_id=27d4e54f75754129bd81de7a6d5a2a05, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.601537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.603780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:56.616783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1129/2248] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:56 2022] PUT /identity/v3/projects/f00dee2ecea141ed9829009f1d2e0ea1/users/27d4e54f75754129bd81de7a6d5a2a05/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:56.627887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.628171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.628445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.655238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] MFA Rules not processed for user `27d4e54f75754129bd81de7a6d5a2a05`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:56.656526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:56.721821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1120/2249] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:56.732390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:56.734109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:56.790993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26805ba3-0963-4f3b-a158-c68070177c87 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:56.794685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b282f4-d241-4e44-bad0-f3c91645b2f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:56.795330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b282f4-d241-4e44-bad0-f3c91645b2f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:56.795618 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b282f4-d241-4e44-bad0-f3c91645b2f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:56.795899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b282f4-d241-4e44-bad0-f3c91645b2f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:56.804112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b282f4-d241-4e44-bad0-f3c91645b2f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:56.805536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b282f4-d241-4e44-bad0-f3c91645b2f7 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:56.815201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1130/2250] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:58.715046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:58.716666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:58.785495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677454b-cb7a-4220-87d9-e851ee84b487 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:58.790117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:58.790998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:58.791289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:58.791570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] PATH_INFO: `/v3/users/2260752f4ad948659ff46b6d06698141` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:58.792771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2260752f4ad948659ff46b6d06698141)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:58.794816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:58.856185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2260752f4ad948659ff46b6d06698141'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:58.867813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2260752f4ad948659ff46b6d06698141'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:58.877701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2260752f4ad948659ff46b6d06698141'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:58.878449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2260752f4ad948659ff46b6d06698141'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:58.895134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2260752f4ad948659ff46b6d06698141'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:58.903025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1121/2251] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:40:58 2022] DELETE /identity/v3/users/2260752f4ad948659ff46b6d06698141 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:58.907531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:58.908338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:58.908485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:58.926788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:58.928005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:58.990453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1131/2252] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:59.016135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.017650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:59.025125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:59.081882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe5d8d-39ee-47b4-9f2b-9f029bf14fd1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.085460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923943c-5936-4f98-9387-3725da54044c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.086167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923943c-5936-4f98-9387-3725da54044c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.086457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923943c-5936-4f98-9387-3725da54044c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.086735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923943c-5936-4f98-9387-3725da54044c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.097671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923943c-5936-4f98-9387-3725da54044c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.107864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923943c-5936-4f98-9387-3725da54044c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.108954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1122/2253] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:40:59.371657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.373409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:59.394710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9f9d-c2a7-4c2e-b6c2-b4145446e4a9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.401492 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.402727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.403171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.403503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] PATH_INFO: `/v3/projects/240538f0531e4d119946702e8a96a720` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.405414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240538f0531e4d119946702e8a96a720)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.407686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.459909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240538f0531e4d119946702e8a96a720', 'request_id': 'req-e321ea98-2eaf-435e-83bc-bb67be22d077'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:59.471106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240538f0531e4d119946702e8a96a720', 'request_id': 'req-e321ea98-2eaf-435e-83bc-bb67be22d077'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:59.479681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240538f0531e4d119946702e8a96a720', 'request_id': 'req-e321ea98-2eaf-435e-83bc-bb67be22d077'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:59.492345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] The token cache is being invalidate because project 240538f0531e4d119946702e8a96a720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:40:59.492853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:59.493521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e321ea98-2eaf-435e-83bc-bb67be22d077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:40:59.495660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1132/2254] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:40:59 2022] DELETE /identity/v3/projects/240538f0531e4d119946702e8a96a720 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:59.502271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f939-97ee-44a3-8c5c-092cfe272a92 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.502653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f939-97ee-44a3-8c5c-092cfe272a92 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.502997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f939-97ee-44a3-8c5c-092cfe272a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.534735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bde0f939-97ee-44a3-8c5c-092cfe272a92 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:59.535962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bde0f939-97ee-44a3-8c5c-092cfe272a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:59.596963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1123/2255] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:40:59.605117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d10e6-626c-46d3-88b1-d0abb3347862 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.605476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d10e6-626c-46d3-88b1-d0abb3347862 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.605857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d10e6-626c-46d3-88b1-d0abb3347862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.629223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4c8d10e6-626c-46d3-88b1-d0abb3347862 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:40:59.630201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4c8d10e6-626c-46d3-88b1-d0abb3347862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:40:59.638792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1133/2256] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:40:59.644266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f939-97ee-44a3-8c5c-092cfe272a92 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.654007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f939-97ee-44a3-8c5c-092cfe272a92 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.659429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.660100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.660386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.660658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.661421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:40:59.661733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.663453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.681246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1124/2257] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:40:59.685545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d10e6-626c-46d3-88b1-d0abb3347862 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.697528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d10e6-626c-46d3-88b1-d0abb3347862 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.702633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.703299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.703579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.703851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.705228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.707144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.717580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:40:59.739930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1134/2258] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:40:59 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:59.741095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.752125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837fafb-850a-4ced-8719-189b4bff1135 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.833298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.834013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.834300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.834571 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.835278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.837202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.860642 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1125/2259] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:40:59 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:40:59.864526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.875302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908cd61-4c33-407a-9575-c958d5c1b536 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.880690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.881384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.881667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.881969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.882700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.884293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.895613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1135/2260] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:59.899724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.910555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2bbc3-362e-49b6-9b85-f41f4a739054 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.915560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035789b8-2855-400c-a676-3ad741482552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.916214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035789b8-2855-400c-a676-3ad741482552 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.916522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035789b8-2855-400c-a676-3ad741482552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.916794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035789b8-2855-400c-a676-3ad741482552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.917542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035789b8-2855-400c-a676-3ad741482552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.919207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035789b8-2855-400c-a676-3ad741482552 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.929016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1126/2261] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:40:59.933149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:40:59.943648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e15f61-b23d-464f-a6a3-2627c3f934b2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:40:59.948240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:40:59.949105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.949387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.949660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] PATH_INFO: `/v3/projects/0c0d3264033d4a129abf8af4c96d10ca/users/22987d0638954a10970f78f145eb3284/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.964512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0d3264033d4a129abf8af4c96d10ca, user_id=22987d0638954a10970f78f145eb3284, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:40:59.967172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:40:59.979342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1136/2262] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:40:59 2022] PUT /identity/v3/projects/0c0d3264033d4a129abf8af4c96d10ca/users/22987d0638954a10970f78f145eb3284/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:40:59.989338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e023ff2-27c9-4729-aa04-c9db17539f15 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:40:59.989625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e023ff2-27c9-4729-aa04-c9db17539f15 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:40:59.989929 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e023ff2-27c9-4729-aa04-c9db17539f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:40:59.996534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:00.018375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2e023ff2-27c9-4729-aa04-c9db17539f15 None None] MFA Rules not processed for user `22987d0638954a10970f78f145eb3284`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:00.019627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2e023ff2-27c9-4729-aa04-c9db17539f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:00.082867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1127/2263] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:40:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:00.093617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:00.095401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:00.153583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4e31a-e33f-48c6-abef-76c1eff39f18 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:00.157167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6b0bb-cd81-4a13-aa17-4e33d54cb56f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:00.157816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6b0bb-cd81-4a13-aa17-4e33d54cb56f service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:00.158398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6b0bb-cd81-4a13-aa17-4e33d54cb56f service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:00.158675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6b0bb-cd81-4a13-aa17-4e33d54cb56f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:00.167465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6b0bb-cd81-4a13-aa17-4e33d54cb56f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:00.168890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6b0bb-cd81-4a13-aa17-4e33d54cb56f service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:00.179164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1137/2264] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:04.691591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:04.692067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:04.692167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:04.713394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] MFA Rules not processed for user `22987d0638954a10970f78f145eb3284`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:04.714625 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:04.722045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1128/2265] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:41:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:10.174757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:14.707659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=33 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:20.053013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409958bd-ee9c-4fc2-98f0-a08bc2cbe583 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:20.053589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409958bd-ee9c-4fc2-98f0-a08bc2cbe583 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:20.053589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409958bd-ee9c-4fc2-98f0-a08bc2cbe583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:20.075305 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-409958bd-ee9c-4fc2-98f0-a08bc2cbe583 None None] MFA Rules not processed for user `9e209f3667f54c2b9bc6eab13010296c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:20.076553 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-409958bd-ee9c-4fc2-98f0-a08bc2cbe583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:20.150352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1138/2266] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:41:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:21.576555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:21.578373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:21.656961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966327c-e2ed-4806-a285-d107268fff9b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:21.661736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:21.662657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.662946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.663222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] PATH_INFO: `/v3/users/27d4e54f75754129bd81de7a6d5a2a05` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.664419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d4e54f75754129bd81de7a6d5a2a05)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:21.666379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:21.711521 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d4e54f75754129bd81de7a6d5a2a05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.718186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d4e54f75754129bd81de7a6d5a2a05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.725395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d4e54f75754129bd81de7a6d5a2a05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.726058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d4e54f75754129bd81de7a6d5a2a05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.738560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d4e54f75754129bd81de7a6d5a2a05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:21.744825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1129/2267] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:21 2022] DELETE /identity/v3/users/27d4e54f75754129bd81de7a6d5a2a05 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:21.749171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.749646 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.749747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.768704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:21.769953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:21.832417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1139/2268] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:21.841776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:21.843238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:21.900915 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88912295-e1a1-4246-8c60-4ccb81e150ce admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:21.904592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24377f1c-1a70-41b2-b4a0-707d6856eab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:21.905244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24377f1c-1a70-41b2-b4a0-707d6856eab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:21.905534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24377f1c-1a70-41b2-b4a0-707d6856eab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:21.905814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24377f1c-1a70-41b2-b4a0-707d6856eab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:21.915129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24377f1c-1a70-41b2-b4a0-707d6856eab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:21.916566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24377f1c-1a70-41b2-b4a0-707d6856eab6 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:21.926526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1130/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:22.201671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:22.203204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:22.216920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f0582-7ba8-4908-ae43-38eb878e7fbc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:22.222042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:22.222922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:22.223208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:22.223485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] PATH_INFO: `/v3/projects/f00dee2ecea141ed9829009f1d2e0ea1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:22.225504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00dee2ecea141ed9829009f1d2e0ea1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:22.227493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:22.285348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00dee2ecea141ed9829009f1d2e0ea1', 'request_id': 'req-761ad035-45d0-42c2-bee6-a93f64791633'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:22.292879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00dee2ecea141ed9829009f1d2e0ea1', 'request_id': 'req-761ad035-45d0-42c2-bee6-a93f64791633'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:22.298508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00dee2ecea141ed9829009f1d2e0ea1', 'request_id': 'req-761ad035-45d0-42c2-bee6-a93f64791633'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:22.309820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] The token cache is being invalidate because project f00dee2ecea141ed9829009f1d2e0ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:22.310146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:22.310777 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-761ad035-45d0-42c2-bee6-a93f64791633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:22.312316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1140/2270] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:22 2022] DELETE /identity/v3/projects/f00dee2ecea141ed9829009f1d2e0ea1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:41:27.419563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:27.420029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:27.420123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:27.445471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] MFA Rules not processed for user `c2c930cfab47464fb784c98f58abbb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:27.448699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:27.472040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:27.526918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1131/2271] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:41:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:32.307008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:37.518838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=15 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:40.399278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab4dd0-ee9b-4c3f-9703-139e4a2d560c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:40.399722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab4dd0-ee9b-4c3f-9703-139e4a2d560c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:40.399844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab4dd0-ee9b-4c3f-9703-139e4a2d560c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:40.459708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e0ab4dd0-ee9b-4c3f-9703-139e4a2d560c None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:40.461019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e0ab4dd0-ee9b-4c3f-9703-139e4a2d560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:40.561939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1141/2272] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:41:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:40.621493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:40.624322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:40.764111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ede3e1-4bb4-4173-8c8c-19d3834a18f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:40.769824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:40.770909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:40.771298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:40.771672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:40.782357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:40.784979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:40.795809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1132/2273] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:41:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:41:41.466890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c1be-99fe-412a-b51e-d20de15378c0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:41.467403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c1be-99fe-412a-b51e-d20de15378c0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:41.467669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581c1be-99fe-412a-b51e-d20de15378c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:41.489125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c581c1be-99fe-412a-b51e-d20de15378c0 None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:41.490746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c581c1be-99fe-412a-b51e-d20de15378c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:41.498878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1142/2274] 10.222.0.25 () {58 vars in 967 bytes} [Fri Jun 17 11:41:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:41.511856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:41.524818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1f0f7-564b-40e1-8910-0a1deefd20f5 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:41.530218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:41.530973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:41.531357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:41.531736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:41.542219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:41.547983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:41.553391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1133/2275] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:41:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:41:41.784320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a4db-15b1-4270-8cd2-c3835b1b6b64 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:41.784615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a4db-15b1-4270-8cd2-c3835b1b6b64 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:41.784900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a4db-15b1-4270-8cd2-c3835b1b6b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:41.803042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3dc9a4db-15b1-4270-8cd2-c3835b1b6b64 None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:41.804262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3dc9a4db-15b1-4270-8cd2-c3835b1b6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:41.811989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1143/2276] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:41:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:41.824922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:41.836704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0fc4-7d4d-4d90-9804-dabf7b1872d4 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:41.843611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:41.844271 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:41.844561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:41.844843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:41.854258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:41.855869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:41.865637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1134/2277] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:41:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:41:42.215645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.215972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.216257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.236066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:42.237631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:42.244910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1144/2278] 10.222.0.25 () {58 vars in 967 bytes} [Fri Jun 17 11:41:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:41:42.257077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:42.269023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b01d9-f0b7-4dae-a126-3ded9f52abae service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:42.274635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27fb1-735d-4a7e-9f53-f537c2d673a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:42.275289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27fb1-735d-4a7e-9f53-f537c2d673a7 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:42.275574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27fb1-735d-4a7e-9f53-f537c2d673a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:42.275875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27fb1-735d-4a7e-9f53-f537c2d673a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:42.285456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27fb1-735d-4a7e-9f53-f537c2d673a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:42.287812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27fb1-735d-4a7e-9f53-f537c2d673a7 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:42.296072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1135/2279] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:41:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:41:46.597016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.598535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:46.691526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802c46b-0914-4dd1-a8f1-98cdc6cda487 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:46.699129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:46.700212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.700550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.701001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] PATH_INFO: `/v3/users/22987d0638954a10970f78f145eb3284` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.703255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22987d0638954a10970f78f145eb3284)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:46.707527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:46.767793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:46.777029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22987d0638954a10970f78f145eb3284'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:46.789545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22987d0638954a10970f78f145eb3284'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:46.802725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22987d0638954a10970f78f145eb3284'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:46.806840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22987d0638954a10970f78f145eb3284'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:46.824202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22987d0638954a10970f78f145eb3284'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:46.834829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1145/2280] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:41:46 2022] DELETE /identity/v3/users/22987d0638954a10970f78f145eb3284 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:46.848945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:46.849421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:46.849604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:46.867801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:41:46.869258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:46.947442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1136/2281] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:41:46.958578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:46.960770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.026648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559f61f-c19f-46ca-9113-cedd6ac83e54 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.030283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.030942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.031227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.031500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.042091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.043530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.054827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1146/2282] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:41:47.381255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:47.382672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:47.396839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fd9c5-ab27-4b1a-b702-d307e27dd32b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:47.407346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:47.409234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:47.409587 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:47.409965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] PATH_INFO: `/v3/projects/0c0d3264033d4a129abf8af4c96d10ca` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:47.411527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0d3264033d4a129abf8af4c96d10ca)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:47.413533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:47.459957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:47.489901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0d3264033d4a129abf8af4c96d10ca', 'request_id': 'req-58fe5c0f-a708-4d34-b356-b7e9b51bceda'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.509367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0d3264033d4a129abf8af4c96d10ca', 'request_id': 'req-58fe5c0f-a708-4d34-b356-b7e9b51bceda'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.516343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0d3264033d4a129abf8af4c96d10ca', 'request_id': 'req-58fe5c0f-a708-4d34-b356-b7e9b51bceda'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.529329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] The token cache is being invalidate because project 0c0d3264033d4a129abf8af4c96d10ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:41:47.529624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.530344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-58fe5c0f-a708-4d34-b356-b7e9b51bceda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:41:47.532006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1137/2283] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:41:47 2022] DELETE /identity/v3/projects/0c0d3264033d4a129abf8af4c96d10ca => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:41:55.419735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:41:55.421919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:55.556915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb9bc0-cff6-4e0c-9df9-d3c077ceab58 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:41:55.613865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:41:55.647832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e863091-324c-4c9e-90e4-b19b66c6a823 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:41:55.648517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863091-324c-4c9e-90e4-b19b66c6a823 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:41:55.648799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863091-324c-4c9e-90e4-b19b66c6a823 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:41:55.649075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863091-324c-4c9e-90e4-b19b66c6a823 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:41:55.651009 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8e863091-324c-4c9e-90e4-b19b66c6a823 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:41:55.663364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e863091-324c-4c9e-90e4-b19b66c6a823 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:41:55.664982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e863091-324c-4c9e-90e4-b19b66c6a823 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:41:55.678934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1147/2284] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:41:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:41:57.526107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:05.673418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=13 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:42:18.439537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:42:18.440029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:42:18.440132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:42:18.459528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] MFA Rules not processed for user `ea8c61c001fe4eafa66c35bc8430f235`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:42:18.460734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:42:18.514773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1138/2285] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:42:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:42:28.508057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=25 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:07.059093 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8af33-2d45-4e4f-b459-d1da4b41ff1c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:07.059580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8af33-2d45-4e4f-b459-d1da4b41ff1c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:07.059665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8af33-2d45-4e4f-b459-d1da4b41ff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:07.077562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d2b8af33-2d45-4e4f-b459-d1da4b41ff1c None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:07.078823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d2b8af33-2d45-4e4f-b459-d1da4b41ff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:07.136660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1148/2286] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:43:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:07.191647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:07.193249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:07.263143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef8996-6146-47ab-b340-671e250d37b1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:07.268683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a4a6-1e01-4118-acca-9874155ee8c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:07.269365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a4a6-1e01-4118-acca-9874155ee8c9 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:07.269655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a4a6-1e01-4118-acca-9874155ee8c9 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:07.269962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a4a6-1e01-4118-acca-9874155ee8c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:07.278500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a4a6-1e01-4118-acca-9874155ee8c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:07.280091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a4a6-1e01-4118-acca-9874155ee8c9 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:07.288988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1139/2287] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:43:17.129509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=25 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:17.286250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=32 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:22.174533 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1dcc-f08c-42fa-9f0b-f3d189e4bee5 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.175095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1dcc-f08c-42fa-9f0b-f3d189e4bee5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.175095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e1dcc-f08c-42fa-9f0b-f3d189e4bee5 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.176555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1149/2288] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:43:22.183058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283e43-a45c-4dba-b8b4-6734339bedb2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.183545 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283e43-a45c-4dba-b8b4-6734339bedb2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.183740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b283e43-a45c-4dba-b8b4-6734339bedb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.241037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6b283e43-a45c-4dba-b8b4-6734339bedb2 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:22.242382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6b283e43-a45c-4dba-b8b4-6734339bedb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:22.323614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1140/2289] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:43:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:22.335324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1dcc-f08c-42fa-9f0b-f3d189e4bee5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:22.337008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e06e1dcc-f08c-42fa-9f0b-f3d189e4bee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:22.419380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e1dcc-f08c-42fa-9f0b-f3d189e4bee5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:22.422999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feacf0-9d26-441b-a7d7-58ec69af23c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f3b6ba74fb49a280735bb8c867d26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:22.423650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feacf0-9d26-441b-a7d7-58ec69af23c9 service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.423990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feacf0-9d26-441b-a7d7-58ec69af23c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.424274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feacf0-9d26-441b-a7d7-58ec69af23c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.436472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feacf0-9d26-441b-a7d7-58ec69af23c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:22.437972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feacf0-9d26-441b-a7d7-58ec69af23c9 service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:22.449132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1150/2290] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:22.515424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af27edf-e76b-42c5-bb19-b785d5a796a7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.515718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af27edf-e76b-42c5-bb19-b785d5a796a7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.515996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af27edf-e76b-42c5-bb19-b785d5a796a7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.517418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1141/2291] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:43:22.522560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af42f8-c82b-4830-bdb5-adc8a14e54de None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.522851 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af42f8-c82b-4830-bdb5-adc8a14e54de None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.523127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af42f8-c82b-4830-bdb5-adc8a14e54de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.542282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-39af42f8-c82b-4830-bdb5-adc8a14e54de None None] MFA Rules not processed for user `b3b35d6d2cd848f4841352fbb58f2053`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:22.543498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-39af42f8-c82b-4830-bdb5-adc8a14e54de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:22.595884 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1151/2292] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:43:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:22.608135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af27edf-e76b-42c5-bb19-b785d5a796a7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:22.609919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7af27edf-e76b-42c5-bb19-b785d5a796a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:22.623167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af27edf-e76b-42c5-bb19-b785d5a796a7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:22.626857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f53d1d-1fed-4695-8b7f-e5e8add4cf6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:22.627512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53d1d-1fed-4695-8b7f-e5e8add4cf6e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.627799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53d1d-1fed-4695-8b7f-e5e8add4cf6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.628074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53d1d-1fed-4695-8b7f-e5e8add4cf6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.635860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f53d1d-1fed-4695-8b7f-e5e8add4cf6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:22.637365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f53d1d-1fed-4695-8b7f-e5e8add4cf6e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:22.645926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1142/2293] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:22.731334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9263eea-0139-4734-9bf1-bcb71493408a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.731655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9263eea-0139-4734-9bf1-bcb71493408a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.731935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9263eea-0139-4734-9bf1-bcb71493408a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.733365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1152/2294] 10.222.0.25 () {54 vars in 921 bytes} [Fri Jun 17 11:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 17 11:43:22.738990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.739282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.739562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.757696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] MFA Rules not processed for user `804a35678c91438eae36f61c1de8c579`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:22.758937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:22.769378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1143/2295] 10.222.0.25 () {58 vars in 1018 bytes} [Fri Jun 17 11:43:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:22.779836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9263eea-0139-4734-9bf1-bcb71493408a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:22.781822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d9263eea-0139-4734-9bf1-bcb71493408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:22.796909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9263eea-0139-4734-9bf1-bcb71493408a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:22.800957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:22.801610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:22.801926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:22.802202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:22.813189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:22.814657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:22.825754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1153/2296] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:26.038302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.039843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:26.104404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8120fb-7ff4-4b18-a6bc-4589eb219f54 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.108969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.109861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.110154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.110436 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] PATH_INFO: `/v3/users/ea8c61c001fe4eafa66c35bc8430f235` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.111624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8c61c001fe4eafa66c35bc8430f235)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.113546 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.155399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:26.161419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8c61c001fe4eafa66c35bc8430f235'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.168653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8c61c001fe4eafa66c35bc8430f235'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.175843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8c61c001fe4eafa66c35bc8430f235'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.176512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8c61c001fe4eafa66c35bc8430f235'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.193898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8f07f41a-912b-4eec-8a3a-f6c10771816e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8c61c001fe4eafa66c35bc8430f235'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.201718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1144/2297] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:43:26 2022] DELETE /identity/v3/users/ea8c61c001fe4eafa66c35bc8430f235 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:26.375277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.386501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acef4-ca36-4723-b892-5228a9c6c069 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.444652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.445580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.445901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.446181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] PATH_INFO: `/v3/projects/5c8c935ef81f487ea6cac72f452bdc5d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.447458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c8c935ef81f487ea6cac72f452bdc5d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.449608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.472032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:26.495819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c8c935ef81f487ea6cac72f452bdc5d', 'request_id': 'req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.503665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c8c935ef81f487ea6cac72f452bdc5d', 'request_id': 'req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.509180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c8c935ef81f487ea6cac72f452bdc5d', 'request_id': 'req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.520097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] The token cache is being invalidate because project 5c8c935ef81f487ea6cac72f452bdc5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:43:26.520389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.521021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c2e0aa18-9a32-4fb0-83c8-0a0ce343147d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:43:26.522620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1154/2298] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:43:26 2022] DELETE /identity/v3/projects/5c8c935ef81f487ea6cac72f452bdc5d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:26.528649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98906895-9067-437e-aaa5-c0f597364ceb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.528965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98906895-9067-437e-aaa5-c0f597364ceb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.529246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98906895-9067-437e-aaa5-c0f597364ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.547835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-98906895-9067-437e-aaa5-c0f597364ceb None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:26.549550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-98906895-9067-437e-aaa5-c0f597364ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:26.617231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1145/2299] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:26.625638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342293be-3f80-4a6c-8a53-19f51a1d0005 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.625982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342293be-3f80-4a6c-8a53-19f51a1d0005 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.626261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342293be-3f80-4a6c-8a53-19f51a1d0005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.647611 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-342293be-3f80-4a6c-8a53-19f51a1d0005 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:26.648826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-342293be-3f80-4a6c-8a53-19f51a1d0005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:26.657229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1155/2300] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:26.661450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98906895-9067-437e-aaa5-c0f597364ceb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.671955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98906895-9067-437e-aaa5-c0f597364ceb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.677510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.678220 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.678508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.678789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.679656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:43:26.679975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.681616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.694926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1146/2301] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:43:26.699350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342293be-3f80-4a6c-8a53-19f51a1d0005 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.708522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342293be-3f80-4a6c-8a53-19f51a1d0005 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.713568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.714261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.714549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.714828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.715527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.717443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.745735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1156/2302] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:26 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:26.749922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.759847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75358ef8-2c43-4e80-a1b6-2f11e1d9ead7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.818794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.819479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.819764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.820040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.820792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.822803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.846397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1147/2303] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:26 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:26.847627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.857832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38026f4c-6404-4fb0-968c-8cb0d3f3ac22 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.863707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.864380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.864666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.864941 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.865671 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.867343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.877969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1157/2304] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:26.881854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.892239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd29992-f4d3-4ce5-b230-06b98ec347f8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.896950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfe394-1fa1-4674-9856-c44c8e7b7a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.897624 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe394-1fa1-4674-9856-c44c8e7b7a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.897940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe394-1fa1-4674-9856-c44c8e7b7a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.898217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfe394-1fa1-4674-9856-c44c8e7b7a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.898954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfe394-1fa1-4674-9856-c44c8e7b7a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.900567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfe394-1fa1-4674-9856-c44c8e7b7a91 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.913016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1148/2305] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:26.917026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:26.927721 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc16ebc-28fd-4971-81d8-71bb7356ad96 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:26.934900 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:26.935894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.936174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.936447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] PATH_INFO: `/v3/projects/21cd7e2263234026bd12c574c137e311/users/0f1328cad6ed4e1b8f446894f81f3fb4/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:26.953694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cd7e2263234026bd12c574c137e311, user_id=0f1328cad6ed4e1b8f446894f81f3fb4, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:26.956468 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:26.977667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1158/2306] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:26 2022] PUT /identity/v3/projects/21cd7e2263234026bd12c574c137e311/users/0f1328cad6ed4e1b8f446894f81f3fb4/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:26.979594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540671b7-170a-4376-b9c5-b1f12f5c715b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:26.979887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540671b7-170a-4376-b9c5-b1f12f5c715b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:26.980165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540671b7-170a-4376-b9c5-b1f12f5c715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:27.001064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-540671b7-170a-4376-b9c5-b1f12f5c715b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:27.002332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-540671b7-170a-4376-b9c5-b1f12f5c715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:27.023513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:27.071601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1149/2307] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:27.081703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:27.083478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:27.144233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c68611-810e-4686-8ec8-7f1a7f7cd2d7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:27.147976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:27.148635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:27.148968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:27.149202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:27.160054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:27.161502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:27.171541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1159/2308] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:32.666520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c11d4f-d9a4-4cab-a237-01f08a9fbc5e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:32.667429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c11d4f-d9a4-4cab-a237-01f08a9fbc5e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:32.667907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c11d4f-d9a4-4cab-a237-01f08a9fbc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:32.697338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-22c11d4f-d9a4-4cab-a237-01f08a9fbc5e None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:32.699243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-22c11d4f-d9a4-4cab-a237-01f08a9fbc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:32.786356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1150/2309] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:43:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:32.815435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:32.818641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:32.913516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:32.944730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b11fb0-6a45-431f-9679-2fa6ffa863a8 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:32.950549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:32.951382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:32.951673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:32.951930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:32.969004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:32.971069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:32.984368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1160/2310] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:43:33.212290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc92ca6-f79f-4389-857f-40523065574a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:33.213248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc92ca6-f79f-4389-857f-40523065574a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:33.213974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc92ca6-f79f-4389-857f-40523065574a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:33.287468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5cc92ca6-f79f-4389-857f-40523065574a None None] MFA Rules not processed for user `0f1328cad6ed4e1b8f446894f81f3fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:33.290214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5cc92ca6-f79f-4389-857f-40523065574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:33.447859 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1151/2311] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:33.452709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:33.468310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb886ad1-e49d-4eca-84c2-89b61a84d3d3 service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:33.475018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:33.476454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:33.477056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:33.477483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:33.478557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:33.480638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:33.520098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1161/2312] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:43:33 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:33.524425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc92ca6-f79f-4389-857f-40523065574a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:33.543286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc92ca6-f79f-4389-857f-40523065574a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:33.685421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:33.686312 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:33.686602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:33.686882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:33.688366 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:33.692948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:33.709731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:33.739978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:33.751822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1152/2313] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:43:33 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:43:33.765335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b12f6-5320-4bb1-a741-6931c31f0a88 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:33.773162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:33.774000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:33.774369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:33.775229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:33.776186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:33.778949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:33.800069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1162/2314] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:33.804652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:33.819294 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d24fd6-d026-4528-aab0-948563a2ef54 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:33.824329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:33.825243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:33.825531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:33.825833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] PATH_INFO: `/v3/projects/83362f3cee7b46e1a1e44e630d2866a2/users/f64877c00a7e4b5483a6940c1c93369d/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:33.867310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:33.869587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:33.878652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83362f3cee7b46e1a1e44e630d2866a2, user_id=f64877c00a7e4b5483a6940c1c93369d, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:33.887095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:33.921348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1153/2315] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:33 2022] PUT /identity/v3/projects/83362f3cee7b46e1a1e44e630d2866a2/users/f64877c00a7e4b5483a6940c1c93369d/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:33.926422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:34.050264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3071c0-b757-4d50-a690-d15a573221f5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:34.056032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:34.056733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:34.057022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:34.057287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:34.058068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:34.059867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:34.075587 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1154/2316] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:43:33 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:43:34.079452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:34.092373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e485f5-5cc3-4f49-9faa-fee39d283fe4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:34.098561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:34.099511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:34.099792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:34.100067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] PATH_INFO: `/v3/projects/83362f3cee7b46e1a1e44e630d2866a2/users/f64877c00a7e4b5483a6940c1c93369d/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:34.102262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=83362f3cee7b46e1a1e44e630d2866a2, user_id=f64877c00a7e4b5483a6940c1c93369d, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:34.105230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:34.137262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1155/2317] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:43:34 2022] PUT /identity/v3/projects/83362f3cee7b46e1a1e44e630d2866a2/users/f64877c00a7e4b5483a6940c1c93369d/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:43:34.190481 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6beecb7-5a13-4c6f-b762-39d037dc564f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:34.314021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f639432-f73a-4292-bb68-dd20a9ab83ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:34.314705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639432-f73a-4292-bb68-dd20a9ab83ba service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:34.314996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639432-f73a-4292-bb68-dd20a9ab83ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:34.315268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f639432-f73a-4292-bb68-dd20a9ab83ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:34.317164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6f639432-f73a-4292-bb68-dd20a9ab83ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:34.426848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:34.463487 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f639432-f73a-4292-bb68-dd20a9ab83ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:34.464944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f639432-f73a-4292-bb68-dd20a9ab83ba service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:34.492208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1163/2318] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:36.666041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:36.668033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:36.863254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf505f-e56d-4eb7-b867-e38adfcce24c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:36.868738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:36.869429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:36.869716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:36.870128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:36.877670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:36.911081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:36.914053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:36.943724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1156/2319] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:42.383113 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5ea8-9366-4ceb-9e0c-9fcb6eb563bc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:42.383617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5ea8-9366-4ceb-9e0c-9fcb6eb563bc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:42.383842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5ea8-9366-4ceb-9e0c-9fcb6eb563bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:42.449909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-453b5ea8-9366-4ceb-9e0c-9fcb6eb563bc None None] MFA Rules not processed for user `f64877c00a7e4b5483a6940c1c93369d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:42.455890 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-453b5ea8-9366-4ceb-9e0c-9fcb6eb563bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:42.559351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1164/2320] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:42.573125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:42.575784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:42.656226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58ff33-c748-4f54-85f7-ef4f996c8c97 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:42.660674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f3b6ba74fb49a280735bb8c867d26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:42.661526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:42.661771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:42.661936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:42.670280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:42.744654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:42.746353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:42.770714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1157/2321] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:42.959997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd576c-8579-4a25-b31a-43e36e7a33db None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:42.960434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd576c-8579-4a25-b31a-43e36e7a33db None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:42.960863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd576c-8579-4a25-b31a-43e36e7a33db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:42.990704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dfdd576c-8579-4a25-b31a-43e36e7a33db None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:43.002590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dfdd576c-8579-4a25-b31a-43e36e7a33db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:43.139149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1165/2322] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:43:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:43.151947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:43.238005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748e44-5794-4551-bdc4-d841c671f1f2 service glance] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:43.245681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:43.246388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:43.246671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:43.246949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:43.261012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:43.262736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:43.275954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1158/2323] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:43:50.022723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.023329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.023436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.047599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] MFA Rules not processed for user `c2c930cfab47464fb784c98f58abbb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:50.050081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.130195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1166/2324] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:50.538088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:50.549494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28089d-a185-4747-88d1-16ae09e59ab3 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:50.553909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d8a4c-00a0-45f1-a6aa-24f7649a10ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:50.554666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d8a4c-00a0-45f1-a6aa-24f7649a10ce service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:50.555047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d8a4c-00a0-45f1-a6aa-24f7649a10ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:50.555422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d8a4c-00a0-45f1-a6aa-24f7649a10ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:50.557387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e14d8a4c-00a0-45f1-a6aa-24f7649a10ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:50.576167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d8a4c-00a0-45f1-a6aa-24f7649a10ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:50.577656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d8a4c-00a0-45f1-a6aa-24f7649a10ce service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:50.587259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1159/2325] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:51.075491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:51.076994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:51.109031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:43:51.200835 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1d1cf-fd57-46c1-b513-3448f017819d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:51.207177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:51.207846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:51.208133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:51.208412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:51.211950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:51.225199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:51.226950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:51.246455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1167/2326] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:43:55.407469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb98ae-bf08-4c3a-a32d-d7533b530e83 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:55.408076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb98ae-bf08-4c3a-a32d-d7533b530e83 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:55.408076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb98ae-bf08-4c3a-a32d-d7533b530e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:55.428434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d0cb98ae-bf08-4c3a-a32d-d7533b530e83 None None] MFA Rules not processed for user `0f1328cad6ed4e1b8f446894f81f3fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:55.429198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0cb98ae-bf08-4c3a-a32d-d7533b530e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:55.442822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1160/2327] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:56.890512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:56.902809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37871f3f-69fa-4139-96ac-b7a3a1df87b2 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:56.907997 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:56.909132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:56.909580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:56.910057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:56.911995 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:56.924747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:56.926499 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:56.936502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1168/2328] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:57.574091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0ec53-152e-43ef-b0b5-af8306fff7d4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:57.574613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0ec53-152e-43ef-b0b5-af8306fff7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:57.574720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0ec53-152e-43ef-b0b5-af8306fff7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:57.591920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-35f0ec53-152e-43ef-b0b5-af8306fff7d4 None None] MFA Rules not processed for user `9e209f3667f54c2b9bc6eab13010296c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:57.593120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-35f0ec53-152e-43ef-b0b5-af8306fff7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:57.650186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1161/2329] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:43:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:57.711423 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:57.713058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:57.729415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d492c6b-4667-49cc-8897-e35511b25470 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:57.736921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:57.737585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:57.737950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:57.738229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:57.740163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:57.804788 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:57.806417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:57.815661 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1169/2330] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:43:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:43:58.667638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274afcc-4d2a-4a2f-ad88-8f80514d3cd6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:58.668146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274afcc-4d2a-4a2f-ad88-8f80514d3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:58.668239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274afcc-4d2a-4a2f-ad88-8f80514d3cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:58.689034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5274afcc-4d2a-4a2f-ad88-8f80514d3cd6 None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:58.690738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5274afcc-4d2a-4a2f-ad88-8f80514d3cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:58.705912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1162/2331] 10.222.0.25 () {58 vars in 967 bytes} [Fri Jun 17 11:43:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:58.713508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:58.725875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a6fac-909c-432b-9d83-3c9c98e32ba0 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:58.732472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:58.733281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:58.733740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:58.734138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:58.744099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:58.745906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:58.756392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1170/2332] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:43:59.382759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:59.383559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:59.384151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:59.408299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:43:59.410424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:43:59.427073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1163/2333] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:43:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:43:59.443970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:43:59.458626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251da8c-ffb5-41ab-bf3d-4f11663fc56a service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:43:59.475353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:43:59.476375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:43:59.478783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:43:59.479425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:43:59.499758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:43:59.505726 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:43:59.520558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1171/2334] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:43:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:44:00.239717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:00.242060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:00.268484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d232a-9bd0-4e69-944a-4826c7c70e61 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:00.283023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:00.283968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:00.284526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:00.284963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:00.293029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:00.322161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:00.327090 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:00.342491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:00.394367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1164/2335] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:01.942049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:01.960140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8470a-3efe-4875-be93-62772bb321ad service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:01.967620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:01.968572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:01.969723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:01.970206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:01.972271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:01.987381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:01.989910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:02.000928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1172/2336] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:10.358132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:11.995933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=76 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:18.536337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:18.548449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a727f-7f56-4534-9a0b-6dfbbace30b4 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:18.555094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:18.555882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:18.556236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:18.556736 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:18.558700 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:18.573100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:18.574854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:18.586806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1165/2337] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:19.722838 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:19.736274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c981a-1f7a-4098-9f6a-6b7296d9eb9f service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:19.742286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4da60-4969-4349-a869-0c93b55b71f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:19.742971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4da60-4969-4349-a869-0c93b55b71f9 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:19.743253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4da60-4969-4349-a869-0c93b55b71f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:19.743720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4da60-4969-4349-a869-0c93b55b71f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:19.745735 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-19b4da60-4969-4349-a869-0c93b55b71f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:44:19.762404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4da60-4969-4349-a869-0c93b55b71f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:19.765044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4da60-4969-4349-a869-0c93b55b71f9 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:19.790264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1173/2338] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:28.582128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:29.774567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:44:34.564873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:44:34.575339 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f436a71-3c5a-4418-b520-0fd7b7a35167 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:44:34.579596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:44:34.580266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:44:34.580555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:44:34.580836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:44:34.590806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:44:34.592289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:44:34.602256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1166/2339] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:44:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:44:44.597699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:00.114700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8b1a-5d79-4dbc-9099-035fd6034c75 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:00.115472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8b1a-5d79-4dbc-9099-035fd6034c75 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:00.115639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc8b1a-5d79-4dbc-9099-035fd6034c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:00.141173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9acc8b1a-5d79-4dbc-9099-035fd6034c75 None None] MFA Rules not processed for user `9b2a8aa8f4724e48a41afce944657ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:00.142604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9acc8b1a-5d79-4dbc-9099-035fd6034c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:00.210660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1174/2340] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:07.432544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:07.465627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0188896-bdfc-41b0-ab79-d2a39feba311 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:07.480840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a8427-babb-43da-bdd2-996129cb9e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:07.484741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a8427-babb-43da-bdd2-996129cb9e3c service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:07.485038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a8427-babb-43da-bdd2-996129cb9e3c service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:07.485314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a8427-babb-43da-bdd2-996129cb9e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:07.488354 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f36a8427-babb-43da-bdd2-996129cb9e3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:07.508860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a8427-babb-43da-bdd2-996129cb9e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:07.510696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a8427-babb-43da-bdd2-996129cb9e3c service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:07.528273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1167/2341] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:09.840223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8b1a-5d79-4dbc-9099-035fd6034c75 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:09.861453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc8b1a-5d79-4dbc-9099-035fd6034c75 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:09.873365 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:09.876526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:09.876852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:09.877453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:09.880006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:09.900043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:09.901743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:09.915678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1175/2342] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:17.519541 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:17.770265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995a4f1-281d-418c-b159-47f76d55f997 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:17.771068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995a4f1-281d-418c-b159-47f76d55f997 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:17.771752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995a4f1-281d-418c-b159-47f76d55f997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:17.797178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0995a4f1-281d-418c-b159-47f76d55f997 None None] MFA Rules not processed for user `3a0f25c7ce194ddbb21d1c8b97d03c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:17.799103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0995a4f1-281d-418c-b159-47f76d55f997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:17.824046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1168/2343] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:18.531489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:18.556218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d5263-83ff-418b-898c-795d56825f6c service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:18.562797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa783588-ec24-4d2e-a845-99ece8a3ce21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:18.563766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa783588-ec24-4d2e-a845-99ece8a3ce21 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:18.563766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa783588-ec24-4d2e-a845-99ece8a3ce21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:18.564837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa783588-ec24-4d2e-a845-99ece8a3ce21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:18.567463 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-aa783588-ec24-4d2e-a845-99ece8a3ce21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:18.584389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa783588-ec24-4d2e-a845-99ece8a3ce21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:18.585896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa783588-ec24-4d2e-a845-99ece8a3ce21 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:18.596122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1176/2344] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:27.790358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:28.591757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=37 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:37.802534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995a4f1-281d-418c-b159-47f76d55f997 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:37.817869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995a4f1-281d-418c-b159-47f76d55f997 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:37.823596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:37.824427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:37.824706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:37.825325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:37.827935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:37.925808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:37.927388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:37.941012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1169/2345] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:41.533917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:41.534578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:41.534578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:41.563557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] MFA Rules not processed for user `c2c930cfab47464fb784c98f58abbb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:41.566909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:41.588033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1177/2346] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:43.990610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:43.992926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:44.020926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b381935-e507-4ef5-8686-454d2cb57292 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:44.032650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:44.034715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:44.035043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:44.035899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:44.039542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:44.065875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:44.070924 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:44.099396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1170/2347] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:44.125183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:44.128394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:44.179066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c9f50-fd59-4536-a736-75d3c7dfc138 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:44.196975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:44.205070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:44.205702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:44.206162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] PATH_INFO: `/v3/users/9e209f3667f54c2b9bc6eab13010296c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:44.207778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e209f3667f54c2b9bc6eab13010296c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:44.210138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:44.290666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e209f3667f54c2b9bc6eab13010296c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.320045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e209f3667f54c2b9bc6eab13010296c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.332674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e209f3667f54c2b9bc6eab13010296c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.333303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e209f3667f54c2b9bc6eab13010296c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.350983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ebd9eb47-82bb-4482-9166-7afd3980c5b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e209f3667f54c2b9bc6eab13010296c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.359955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1178/2348] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:45:44 2022] DELETE /identity/v3/users/9e209f3667f54c2b9bc6eab13010296c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:44.665680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:44.685420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cebf99-5eb4-4ff8-ae3d-b2c1b81bc91c service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:44.806523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:44.808039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:44.808522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:44.808993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] PATH_INFO: `/v3/projects/b54e176d82d943af98e5f81ee153caab` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:44.811109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b54e176d82d943af98e5f81ee153caab)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:44.814735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:44.902141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b54e176d82d943af98e5f81ee153caab', 'request_id': 'req-eea4b2a9-8a50-4533-afa6-6db127ea0f54'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.912509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b54e176d82d943af98e5f81ee153caab', 'request_id': 'req-eea4b2a9-8a50-4533-afa6-6db127ea0f54'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.924308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b54e176d82d943af98e5f81ee153caab', 'request_id': 'req-eea4b2a9-8a50-4533-afa6-6db127ea0f54'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.953435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] The token cache is being invalidate because project b54e176d82d943af98e5f81ee153caab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:45:44.953878 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.954980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eea4b2a9-8a50-4533-afa6-6db127ea0f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:44.956967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1171/2349] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:45:44 2022] DELETE /identity/v3/projects/b54e176d82d943af98e5f81ee153caab => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:44.964895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87512aa5-e5af-40be-ae88-4e1084669b0b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:44.965405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87512aa5-e5af-40be-ae88-4e1084669b0b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:44.965937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87512aa5-e5af-40be-ae88-4e1084669b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:44.999212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-87512aa5-e5af-40be-ae88-4e1084669b0b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:45.001288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-87512aa5-e5af-40be-ae88-4e1084669b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:45.109115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1179/2350] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:45.118840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa97b-9d5b-4770-9a4f-80053eee0720 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.119825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa97b-9d5b-4770-9a4f-80053eee0720 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.119825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa97b-9d5b-4770-9a4f-80053eee0720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.141409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-20ffa97b-9d5b-4770-9a4f-80053eee0720 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:45.142780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-20ffa97b-9d5b-4770-9a4f-80053eee0720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:45.151955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1172/2351] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:45:45.156826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87512aa5-e5af-40be-ae88-4e1084669b0b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:45.168261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87512aa5-e5af-40be-ae88-4e1084669b0b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:45.175459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:45.176155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.177139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.177458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.178270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:45:45.178585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:45.180624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:45.206395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1180/2352] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:45:45.207863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffa97b-9d5b-4770-9a4f-80053eee0720 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:45.224483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:45.233246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffa97b-9d5b-4770-9a4f-80053eee0720 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:45.244502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:45.246770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.247195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.247378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.248755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:45.252690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:45.322180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1173/2353] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:45:45 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:45:45.324940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:45.344176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0705ac-0a3e-4d31-b937-14293a4b4c93 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:45.439251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:45.460963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:45.461725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.462198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.462632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.463486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:45.465706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:45.498337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1181/2354] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:45:45 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:45.504369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:45.534352 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b0752-bd52-48c9-90b5-1d4504c6314a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:45.545799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:45.547188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.547909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.548430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.550128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:45.552383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:45.566835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1174/2355] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:45.573253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:45.594409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b62c2-7705-4c97-ab9a-af28bfbb478f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:45.602903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4c345-9f0c-4d02-b930-8973c201f23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:45.604069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c345-9f0c-4d02-b930-8973c201f23a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.604565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c345-9f0c-4d02-b930-8973c201f23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.605248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c345-9f0c-4d02-b930-8973c201f23a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.606507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4c345-9f0c-4d02-b930-8973c201f23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:45.610070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4c345-9f0c-4d02-b930-8973c201f23a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:45.633917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1182/2356] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:45.637544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:45.658128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171834f-6a83-4fb0-9d90-6dd2a3a71334 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:45.670084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:45.671102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.671386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.671665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] PATH_INFO: `/v3/projects/43043d19d5534383ac1fa4a877a35c72/users/4878e4b276934a81a1cdfc0e2934da8c/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.708258 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43043d19d5534383ac1fa4a877a35c72, user_id=4878e4b276934a81a1cdfc0e2934da8c, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:45.713825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:45.739857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1175/2357] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:45 2022] PUT /identity/v3/projects/43043d19d5534383ac1fa4a877a35c72/users/4878e4b276934a81a1cdfc0e2934da8c/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:45.755781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014568-5b01-4d00-81ae-b08c3b229217 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:45.756278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014568-5b01-4d00-81ae-b08c3b229217 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:45.756693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014568-5b01-4d00-81ae-b08c3b229217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:45.792539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-af014568-5b01-4d00-81ae-b08c3b229217 None None] MFA Rules not processed for user `4878e4b276934a81a1cdfc0e2934da8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:45.794125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-af014568-5b01-4d00-81ae-b08c3b229217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:45.907662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1183/2358] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:45.914219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:46.067005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5cee2-2c29-4526-b53f-e40da409519d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:46.072603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:46.073307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.074187 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.074485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:46.075237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:46.077226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:46.114284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1176/2359] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:45:45 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:45:46.121178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014568-5b01-4d00-81ae-b08c3b229217 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:46.141269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014568-5b01-4d00-81ae-b08c3b229217 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:46.246804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:46.248155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.248622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.249084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:46.250348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:46.254210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:46.289947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1184/2360] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:45:46 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:46.294925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:46.312448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca544c3-e22d-4d4a-b237-7f8ad5c555a2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:46.318311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:46.319093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.319454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.319997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:46.321470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:46.324480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:46.348990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1177/2361] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:46.350976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:46.365737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15ee11-4dc4-4965-838f-e892b4787b9b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:46.373591 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:46.375172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.375752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.376260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] PATH_INFO: `/v3/projects/67eb01f2480944f782c80bd11a659d63/users/af3081959b5c402ca772ee93db369cc5/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:46.398954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67eb01f2480944f782c80bd11a659d63, user_id=af3081959b5c402ca772ee93db369cc5, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:46.402391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:46.422649 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1185/2362] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:46 2022] PUT /identity/v3/projects/67eb01f2480944f782c80bd11a659d63/users/af3081959b5c402ca772ee93db369cc5/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:46.427560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:46.439150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b48d86b-31f0-4eb6-83a9-7dd230579816 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:46.539710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41500f8-d2d8-4b36-a275-d2f12f4a582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:46.540834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41500f8-d2d8-4b36-a275-d2f12f4a582b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.541335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41500f8-d2d8-4b36-a275-d2f12f4a582b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.541644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41500f8-d2d8-4b36-a275-d2f12f4a582b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:46.543652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41500f8-d2d8-4b36-a275-d2f12f4a582b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:46.547350 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41500f8-d2d8-4b36-a275-d2f12f4a582b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:46.571762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1178/2363] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:46.573520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:46.591240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe5c5c-9f88-4eb8-a78e-e8ecb4dfd1d7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:46.600508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:46.601736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.602055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.602631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] PATH_INFO: `/v3/projects/67eb01f2480944f782c80bd11a659d63/users/af3081959b5c402ca772ee93db369cc5/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:46.607283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67eb01f2480944f782c80bd11a659d63, user_id=af3081959b5c402ca772ee93db369cc5, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:46.611393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:46.654849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1186/2364] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:46 2022] PUT /identity/v3/projects/67eb01f2480944f782c80bd11a659d63/users/af3081959b5c402ca772ee93db369cc5/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:46.658367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c02a6-a9a6-4d50-9752-a08787af1429 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.658658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c02a6-a9a6-4d50-9752-a08787af1429 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.658937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c02a6-a9a6-4d50-9752-a08787af1429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:46.684675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:46.706836 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0a2c02a6-a9a6-4d50-9752-a08787af1429 None None] MFA Rules not processed for user `af3081959b5c402ca772ee93db369cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:46.708558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0a2c02a6-a9a6-4d50-9752-a08787af1429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:46.838815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1179/2365] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:45:46.858402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:46.863787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:46.889784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:46.976831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd00b-47ce-424f-8d85-ee2fec57fec7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:46.985315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95856bfb-cdb4-4eb5-8c6e-25a306fce867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:46.986438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95856bfb-cdb4-4eb5-8c6e-25a306fce867 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:46.986967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95856bfb-cdb4-4eb5-8c6e-25a306fce867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:46.987425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95856bfb-cdb4-4eb5-8c6e-25a306fce867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:47.006436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95856bfb-cdb4-4eb5-8c6e-25a306fce867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:47.008524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95856bfb-cdb4-4eb5-8c6e-25a306fce867 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:47.038626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1187/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:47.640803 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c02a6-a9a6-4d50-9752-a08787af1429 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:47.733520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c02a6-a9a6-4d50-9752-a08787af1429 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:47.742030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:47.743577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:47.744371 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:47.744867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] PATH_INFO: `/v3/users/4878e4b276934a81a1cdfc0e2934da8c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:47.748454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4878e4b276934a81a1cdfc0e2934da8c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:47.753020 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:47.835300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4878e4b276934a81a1cdfc0e2934da8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:47.848958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4878e4b276934a81a1cdfc0e2934da8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:47.861508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4878e4b276934a81a1cdfc0e2934da8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:47.862535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4878e4b276934a81a1cdfc0e2934da8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:47.884551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4878e4b276934a81a1cdfc0e2934da8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:47.895975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1180/2367] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:45:47 2022] DELETE /identity/v3/users/4878e4b276934a81a1cdfc0e2934da8c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:47.902624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:47.903397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:47.903397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:47.929727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:47.931609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:48.026697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1188/2368] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:48.044033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:48.046632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:48.064890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:48.139412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53ca19-fca9-4874-aa39-b63183eddd38 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:48.145003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:48.145894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:48.146283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:48.147361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:48.165448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:48.168592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:48.187279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1181/2369] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:48.781876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:48.784165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:48.809603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e5fce-94a1-4d86-ab7d-f886df890a6b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:48.817665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:48.819195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:48.819702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:48.820183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] PATH_INFO: `/v3/projects/43043d19d5534383ac1fa4a877a35c72` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:48.822723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43043d19d5534383ac1fa4a877a35c72)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:48.826057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:48.890084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43043d19d5534383ac1fa4a877a35c72', 'request_id': 'req-336a6724-5fba-4f55-b1c6-4729d04233f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:48.898723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43043d19d5534383ac1fa4a877a35c72', 'request_id': 'req-336a6724-5fba-4f55-b1c6-4729d04233f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:48.906160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43043d19d5534383ac1fa4a877a35c72', 'request_id': 'req-336a6724-5fba-4f55-b1c6-4729d04233f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:48.920382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] The token cache is being invalidate because project 43043d19d5534383ac1fa4a877a35c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:45:48.920680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:48.921869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:48.924667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1189/2370] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:45:48 2022] DELETE /identity/v3/projects/43043d19d5534383ac1fa4a877a35c72 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:45:48.929713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:48.942176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54305ca4-ca20-478c-88a0-4bc8a29723c5 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:49.028213 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:49.029485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:49.030000 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:49.030387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] PATH_INFO: `/v3/users/af3081959b5c402ca772ee93db369cc5` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:49.031963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3081959b5c402ca772ee93db369cc5)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:49.034132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:49.106534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3081959b5c402ca772ee93db369cc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.118964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3081959b5c402ca772ee93db369cc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.131215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3081959b5c402ca772ee93db369cc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.132237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3081959b5c402ca772ee93db369cc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.154529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5b70d65c-2955-4f12-a807-09220c0a3871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3081959b5c402ca772ee93db369cc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.161720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1182/2371] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:45:48 2022] DELETE /identity/v3/users/af3081959b5c402ca772ee93db369cc5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:49.602348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:49.604337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:49.630030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6724-5fba-4f55-b1c6-4729d04233f2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:49.688706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:49.717886 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:49.719317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:49.719821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:49.720263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] PATH_INFO: `/v3/projects/67eb01f2480944f782c80bd11a659d63` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:49.722505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67eb01f2480944f782c80bd11a659d63)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:49.726189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:49.804871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67eb01f2480944f782c80bd11a659d63', 'request_id': 'req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.816753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67eb01f2480944f782c80bd11a659d63', 'request_id': 'req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.827543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67eb01f2480944f782c80bd11a659d63', 'request_id': 'req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.848088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] The token cache is being invalidate because project 67eb01f2480944f782c80bd11a659d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:45:49.848597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.849694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-79fceaf1-1e89-43b6-8f78-90b88bc96ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:49.851934 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1190/2372] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:45:49 2022] DELETE /identity/v3/projects/67eb01f2480944f782c80bd11a659d63 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:49.861764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5172b-aa35-4761-a462-86d7cb6ababe None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:49.862420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5172b-aa35-4761-a462-86d7cb6ababe None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:49.862930 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5172b-aa35-4761-a462-86d7cb6ababe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:49.896990 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2fa5172b-aa35-4761-a462-86d7cb6ababe None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:49.898493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2fa5172b-aa35-4761-a462-86d7cb6ababe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:49.948112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:50.005958 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1183/2373] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:50.010411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb1aa2-c5b1-471c-9d1d-510a5407df71 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.011268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb1aa2-c5b1-471c-9d1d-510a5407df71 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.011795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb1aa2-c5b1-471c-9d1d-510a5407df71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.044178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b2fb1aa2-c5b1-471c-9d1d-510a5407df71 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:50.046092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b2fb1aa2-c5b1-471c-9d1d-510a5407df71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:50.060532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1191/2374] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:50.065905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5172b-aa35-4761-a462-86d7cb6ababe None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.081359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5172b-aa35-4761-a462-86d7cb6ababe None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.089418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.090627 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.091110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.091610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.093039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:45:50.093604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.096661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:50.122022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1184/2375] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:45:50.123559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb1aa2-c5b1-471c-9d1d-510a5407df71 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.140932 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb1aa2-c5b1-471c-9d1d-510a5407df71 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.149176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.150371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.150840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.151294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.152630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.156512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:50.200608 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1192/2376] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:45:50 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:50.202977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.217228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcb73d-1149-40e8-a17b-ad82ee47844a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.306464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.307762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.308968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.309463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.310795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.316146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:50.351613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1185/2377] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:45:50 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:50.356255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.366863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec702fdc-c008-49f2-b58e-3b7d98fbcf86 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.374830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.376110 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.376610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.377092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.378506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.380669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:50.392770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1193/2378] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:50.397139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.410424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7265-796b-4a4b-91a5-622fe3ea3b77 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.418861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928df348-a951-49fa-8d09-a487390d6067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.420113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928df348-a951-49fa-8d09-a487390d6067 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.420612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928df348-a951-49fa-8d09-a487390d6067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.421140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928df348-a951-49fa-8d09-a487390d6067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.422535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928df348-a951-49fa-8d09-a487390d6067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.425702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928df348-a951-49fa-8d09-a487390d6067 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:50.445488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1186/2379] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:50.451668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.463764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803a81e-b96e-4f8f-ac5e-4110306f7f47 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.469597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.470666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.470960 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.471235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] PATH_INFO: `/v3/projects/b4f0290ab36f4081be4f3ec3642cf7e3/users/575f3e61ab18469aac6afd72668aafc4/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.489797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f0290ab36f4081be4f3ec3642cf7e3, user_id=575f3e61ab18469aac6afd72668aafc4, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.492539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:50.517051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1194/2380] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:50 2022] PUT /identity/v3/projects/b4f0290ab36f4081be4f3ec3642cf7e3/users/575f3e61ab18469aac6afd72668aafc4/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:50.532504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5622b-286a-45d9-bf23-4e3bf48bcb88 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.532844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5622b-286a-45d9-bf23-4e3bf48bcb88 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.533180 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5622b-286a-45d9-bf23-4e3bf48bcb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.572898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d4a5622b-286a-45d9-bf23-4e3bf48bcb88 None None] MFA Rules not processed for user `575f3e61ab18469aac6afd72668aafc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:50.574325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d4a5622b-286a-45d9-bf23-4e3bf48bcb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:50.691817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1187/2381] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:50.699420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.804889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e282-fbd7-4501-ba09-fe67c0f3ecf1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.814770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.815924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.816422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.816922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.818270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.822003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:50.843317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:50.868842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1195/2382] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:45:50 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:50.873123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5622b-286a-45d9-bf23-4e3bf48bcb88 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:50.885609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5622b-286a-45d9-bf23-4e3bf48bcb88 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:50.942373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:50.988024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:50.989151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:50.989491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:50.989983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:50.991097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:50.994046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:51.017879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1188/2383] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:45:50 2022] POST /identity/v3/users => generated 298 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:45:51.022126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:51.034099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a98d8-6d54-4574-84da-bb66a7e3371d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:51.039012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:51.039691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:51.039982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:51.040261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:51.041140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:51.042825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:51.054903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1196/2384] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:51.061641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:51.076086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d20414-18e6-4740-b58f-23bcad1a871d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:51.083226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:51.084649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:51.086810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:51.086810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] PATH_INFO: `/v3/projects/bb81de91e0ff4f37972ab7a1487ab44d/users/de0b8a83963b4613a67ab36761204f35/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:51.103605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb81de91e0ff4f37972ab7a1487ab44d, user_id=de0b8a83963b4613a67ab36761204f35, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:51.106369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:51.129470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1189/2385] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:51 2022] PUT /identity/v3/projects/bb81de91e0ff4f37972ab7a1487ab44d/users/de0b8a83963b4613a67ab36761204f35/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:51.135148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:51.146606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729a1c-105a-417e-bc2c-77df6b3edd54 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:51.209386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99b769-a4ce-4168-aefe-13ffd3d8d9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:51.210417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b769-a4ce-4168-aefe-13ffd3d8d9bb admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:51.210692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b769-a4ce-4168-aefe-13ffd3d8d9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:51.211013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99b769-a4ce-4168-aefe-13ffd3d8d9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:51.211822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99b769-a4ce-4168-aefe-13ffd3d8d9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:51.213789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99b769-a4ce-4168-aefe-13ffd3d8d9bb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:51.225261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1197/2386] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:51.230603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:51.243266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b38db-d5f9-439c-bdf7-992ceaa1197c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:51.249182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:51.250514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:51.250846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:51.251171 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] PATH_INFO: `/v3/projects/bb81de91e0ff4f37972ab7a1487ab44d/users/de0b8a83963b4613a67ab36761204f35/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:51.253325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb81de91e0ff4f37972ab7a1487ab44d, user_id=de0b8a83963b4613a67ab36761204f35, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:51.256524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:51.270316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1190/2387] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:51 2022] PUT /identity/v3/projects/bb81de91e0ff4f37972ab7a1487ab44d/users/de0b8a83963b4613a67ab36761204f35/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:51.286765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dec73f-0e61-4042-b0ca-171c2d7a9e5d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:51.287061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dec73f-0e61-4042-b0ca-171c2d7a9e5d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:51.287321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dec73f-0e61-4042-b0ca-171c2d7a9e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:51.323075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b4dec73f-0e61-4042-b0ca-171c2d7a9e5d None None] MFA Rules not processed for user `de0b8a83963b4613a67ab36761204f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:51.325014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b4dec73f-0e61-4042-b0ca-171c2d7a9e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:51.419831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1198/2388] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:51.431817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:51.434887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:51.505872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a45ed9-d99c-49ed-ac76-e5bbb4b5698c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:51.517559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:51.518272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:51.518560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:51.518887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:51.529657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:51.531337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:51.543238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1191/2389] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:51.673563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dec73f-0e61-4042-b0ca-171c2d7a9e5d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:51.689582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dec73f-0e61-4042-b0ca-171c2d7a9e5d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:51.693638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61735f0-37e3-4d81-8352-72de38c43858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:51.694342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61735f0-37e3-4d81-8352-72de38c43858 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:51.694628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61735f0-37e3-4d81-8352-72de38c43858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:51.694905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61735f0-37e3-4d81-8352-72de38c43858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:51.773991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61735f0-37e3-4d81-8352-72de38c43858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:51.775352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61735f0-37e3-4d81-8352-72de38c43858 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:51.781184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:51.788041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1199/2390] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:54.788375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:54.888380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098d7e2-9184-4435-bd41-c75b73d2f745 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:54.893507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:54.894459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:54.894741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:54.895018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] PATH_INFO: `/v3/users/575f3e61ab18469aac6afd72668aafc4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:54.896344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=575f3e61ab18469aac6afd72668aafc4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:54.898365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:54.945204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:54.974674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '575f3e61ab18469aac6afd72668aafc4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:54.986893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '575f3e61ab18469aac6afd72668aafc4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.001101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '575f3e61ab18469aac6afd72668aafc4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.002313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '575f3e61ab18469aac6afd72668aafc4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.022635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '575f3e61ab18469aac6afd72668aafc4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.032357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1192/2391] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:45:54 2022] DELETE /identity/v3/users/575f3e61ab18469aac6afd72668aafc4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:55.037229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:55.037767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:55.037908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:55.065316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:55.066655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:55.172142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1200/2392] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:55.187042 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:55.188988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:55.280060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5e43d-ceea-4877-b75f-5132ac398b05 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:55.285427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:55.286463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:55.286907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:55.287375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:55.302275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:55.304432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:55.321630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1193/2393] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:45:55.748576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:55.750983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:55.773183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741518e-0e2c-4cfd-8d5c-9483f648546a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:55.782053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:55.783594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:55.784109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:55.784622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] PATH_INFO: `/v3/projects/b4f0290ab36f4081be4f3ec3642cf7e3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:55.786540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f0290ab36f4081be4f3ec3642cf7e3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:55.788963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:55.874345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f0290ab36f4081be4f3ec3642cf7e3', 'request_id': 'req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.884322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f0290ab36f4081be4f3ec3642cf7e3', 'request_id': 'req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.893434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f0290ab36f4081be4f3ec3642cf7e3', 'request_id': 'req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.909625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] The token cache is being invalidate because project b4f0290ab36f4081be4f3ec3642cf7e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:45:55.910297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.911086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:55.913111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1201/2394] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:45:55 2022] DELETE /identity/v3/projects/b4f0290ab36f4081be4f3ec3642cf7e3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:55.919252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:55.936248 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063232d7-665c-4883-83df-da8091cdff35 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:56.026635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:56.028236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:56.028979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:56.029389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] PATH_INFO: `/v3/users/de0b8a83963b4613a67ab36761204f35` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:56.031325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de0b8a83963b4613a67ab36761204f35)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:56.034767 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:56.141634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de0b8a83963b4613a67ab36761204f35'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.144326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:56.152518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de0b8a83963b4613a67ab36761204f35'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.162504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de0b8a83963b4613a67ab36761204f35'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.163326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de0b8a83963b4613a67ab36761204f35'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.181939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0288947a-dc60-4e9a-b0a6-fb51c0aa2b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de0b8a83963b4613a67ab36761204f35'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.192165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1194/2395] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:45:55 2022] DELETE /identity/v3/users/de0b8a83963b4613a67ab36761204f35 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:45:56.577120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:56.579239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:56.602460 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d24dc-23ad-4edf-a8d8-ee262e8d142a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:56.683727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:56.684988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:56.685465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:56.685922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] PATH_INFO: `/v3/projects/bb81de91e0ff4f37972ab7a1487ab44d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:56.687949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb81de91e0ff4f37972ab7a1487ab44d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:56.691640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:56.743629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:56.772551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb81de91e0ff4f37972ab7a1487ab44d', 'request_id': 'req-95899472-bd01-4880-87d5-1d284de00696'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.782183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb81de91e0ff4f37972ab7a1487ab44d', 'request_id': 'req-95899472-bd01-4880-87d5-1d284de00696'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.792933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb81de91e0ff4f37972ab7a1487ab44d', 'request_id': 'req-95899472-bd01-4880-87d5-1d284de00696'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.821332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] The token cache is being invalidate because project bb81de91e0ff4f37972ab7a1487ab44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:45:56.821829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.823793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95899472-bd01-4880-87d5-1d284de00696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:56.827147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1202/2396] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:45:56 2022] DELETE /identity/v3/projects/bb81de91e0ff4f37972ab7a1487ab44d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:56.833922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d6c1c-4395-46a9-9d6b-a486186c0094 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:56.834388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d6c1c-4395-46a9-9d6b-a486186c0094 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:56.834860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d6c1c-4395-46a9-9d6b-a486186c0094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:56.865113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f06d6c1c-4395-46a9-9d6b-a486186c0094 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:56.866503 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f06d6c1c-4395-46a9-9d6b-a486186c0094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:56.974727 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1195/2397] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:56.986124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da9692-eed5-4d3c-866e-220e42d5a589 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:56.986988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da9692-eed5-4d3c-866e-220e42d5a589 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:56.987307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da9692-eed5-4d3c-866e-220e42d5a589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.021670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-94da9692-eed5-4d3c-866e-220e42d5a589 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:57.023386 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-94da9692-eed5-4d3c-866e-220e42d5a589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:57.035229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1203/2398] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:57.041707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d6c1c-4395-46a9-9d6b-a486186c0094 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.057644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d6c1c-4395-46a9-9d6b-a486186c0094 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:57.064927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:57.065856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.066230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.066713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.068060 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:45:57.068605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:57.072064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:57.095412 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1196/2399] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:45:57.102239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da9692-eed5-4d3c-866e-220e42d5a589 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.119710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da9692-eed5-4d3c-866e-220e42d5a589 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:57.129183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:57.130357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.130850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.131341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.132655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:57.136512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:57.188236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1204/2400] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:45:57 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:57.193150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.208639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237a231-c9dd-428f-832d-ad564dc70e56 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:57.312769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:57.314133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.314726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.315270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.316712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:57.320431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:57.352353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1197/2401] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:45:57 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:57.357737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.372526 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e4e8-d725-4f01-9919-30a303e99296 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:57.377543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:57.378552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.378908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.379443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.380297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:57.382016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:57.395824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1205/2402] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:45:57.400523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.414186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730336e5-ae12-44ba-b00a-d3e587cf463b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:57.420173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5a786-f77e-407e-8c97-b3d654357c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:57.421173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5a786-f77e-407e-8c97-b3d654357c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.421536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5a786-f77e-407e-8c97-b3d654357c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.421938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5a786-f77e-407e-8c97-b3d654357c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.422946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5a786-f77e-407e-8c97-b3d654357c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:57.424787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5a786-f77e-407e-8c97-b3d654357c2f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:57.449819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1198/2403] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:45:57.452979 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.471709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec27c-c5f8-4ffa-b200-ad7fd0dc4034 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:57.478332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:57.480344 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.480780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.481262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] PATH_INFO: `/v3/projects/87ab6a8b295d4227b776fc4a8e95bf6c/users/f0cc1e554fbf46c2a97c00353b21e327/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.501891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ab6a8b295d4227b776fc4a8e95bf6c, user_id=f0cc1e554fbf46c2a97c00353b21e327, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:57.504539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:57.523874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1206/2404] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:57 2022] PUT /identity/v3/projects/87ab6a8b295d4227b776fc4a8e95bf6c/users/f0cc1e554fbf46c2a97c00353b21e327/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:57.536154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388552b3-542e-4879-a1d9-f7b12e48aea1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.536475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388552b3-542e-4879-a1d9-f7b12e48aea1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.536774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388552b3-542e-4879-a1d9-f7b12e48aea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.572154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-388552b3-542e-4879-a1d9-f7b12e48aea1 None None] MFA Rules not processed for user `f0cc1e554fbf46c2a97c00353b21e327`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:57.574021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-388552b3-542e-4879-a1d9-f7b12e48aea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:57.612275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:57.709126 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1199/2405] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:45:57.726523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.729045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:57.831413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049a273-e999-404f-a251-a145abd106d8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:57.839248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f0abe-0cff-4503-ae48-2b0e074c144a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:57.840419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f0abe-0cff-4503-ae48-2b0e074c144a service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:57.840921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f0abe-0cff-4503-ae48-2b0e074c144a service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:57.841415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f0abe-0cff-4503-ae48-2b0e074c144a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:57.853667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f0abe-0cff-4503-ae48-2b0e074c144a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:57.856001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f0abe-0cff-4503-ae48-2b0e074c144a service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:57.866816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1207/2406] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:45:57.893459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388552b3-542e-4879-a1d9-f7b12e48aea1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:57.992653 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388552b3-542e-4879-a1d9-f7b12e48aea1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:58.000305 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:58.002032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:58.002679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:58.003168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] PATH_INFO: `/v3/users/f0cc1e554fbf46c2a97c00353b21e327` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:58.005580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cc1e554fbf46c2a97c00353b21e327)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:58.009349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:58.090948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cc1e554fbf46c2a97c00353b21e327'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:58.101435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cc1e554fbf46c2a97c00353b21e327'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:58.118255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cc1e554fbf46c2a97c00353b21e327'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:58.119282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cc1e554fbf46c2a97c00353b21e327'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:58.144689 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cc1e554fbf46c2a97c00353b21e327'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:58.156311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1200/2407] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:45:57 2022] DELETE /identity/v3/users/f0cc1e554fbf46c2a97c00353b21e327 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:58.163812 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:58.164351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:58.164869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:58.194615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:58.196396 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:58.269147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:58.293891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1208/2408] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:45:58.309070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:58.311495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:58.418988 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:45:58.439937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dca50f-10ea-4d8f-a83e-4a7b364bae16 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:58.445132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b4882-14b6-45e9-a56f-688d590688ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:58.446064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b4882-14b6-45e9-a56f-688d590688ba service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:58.446549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b4882-14b6-45e9-a56f-688d590688ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:58.447024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b4882-14b6-45e9-a56f-688d590688ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:58.461269 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b4882-14b6-45e9-a56f-688d590688ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:58.462785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b4882-14b6-45e9-a56f-688d590688ba service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:58.478295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1201/2409] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:45:58.952615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:58.955255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:58.986506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6370f-7e4a-4dc2-a539-df7725f88ecf None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:58.997642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:59.004283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.004283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.004283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] PATH_INFO: `/v3/projects/87ab6a8b295d4227b776fc4a8e95bf6c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.004283 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ab6a8b295d4227b776fc4a8e95bf6c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:59.007264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:59.074652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ab6a8b295d4227b776fc4a8e95bf6c', 'request_id': 'req-b03f0297-cb82-4850-8d64-70ec78703136'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:59.088405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ab6a8b295d4227b776fc4a8e95bf6c', 'request_id': 'req-b03f0297-cb82-4850-8d64-70ec78703136'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:59.099912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ab6a8b295d4227b776fc4a8e95bf6c', 'request_id': 'req-b03f0297-cb82-4850-8d64-70ec78703136'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:59.120427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] The token cache is being invalidate because project 87ab6a8b295d4227b776fc4a8e95bf6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:45:59.120933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:59.122059 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b03f0297-cb82-4850-8d64-70ec78703136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:45:59.125155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1209/2410] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:45:58 2022] DELETE /identity/v3/projects/87ab6a8b295d4227b776fc4a8e95bf6c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:59.132701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5073694-1192-42ec-9be8-f4545cc245d9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.133329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5073694-1192-42ec-9be8-f4545cc245d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.133655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5073694-1192-42ec-9be8-f4545cc245d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.165790 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a5073694-1192-42ec-9be8-f4545cc245d9 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:59.168030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5073694-1192-42ec-9be8-f4545cc245d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:59.285189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1202/2411] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:59.287922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c84d0-9afb-4698-b2fd-911495137c20 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.288501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c84d0-9afb-4698-b2fd-911495137c20 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.288906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c84d0-9afb-4698-b2fd-911495137c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.317572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-369c84d0-9afb-4698-b2fd-911495137c20 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:59.318950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-369c84d0-9afb-4698-b2fd-911495137c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:45:59.327931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1210/2412] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:45:59.337444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5073694-1192-42ec-9be8-f4545cc245d9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:59.357673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5073694-1192-42ec-9be8-f4545cc245d9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:59.364207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:59.365217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.365666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.366067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.366918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:45:59.367328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:59.369586 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:59.386883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1203/2413] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:45:59.394190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c84d0-9afb-4698-b2fd-911495137c20 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:59.411271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c84d0-9afb-4698-b2fd-911495137c20 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:59.428589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:59.430011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.430604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.431155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.433231 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:59.437407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:59.491503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1211/2414] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:45:59 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:45:59.493077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:59.524552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82f74-2b28-432b-aeec-dff2a1fe6ef4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:59.665121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:59.666054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.666344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.666703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.667428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:59.669609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:59.699861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1204/2415] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:45:59 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:45:59.704820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:59.722833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfddf2-6da4-4f33-ae86-d5fb1f3444d3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:59.730464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:59.731546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.732370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.732670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.733510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:59.735222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:59.748308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1212/2416] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:59.753864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:59.779585 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbac949-3c7b-49c1-96dd-c44941abacf5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:59.787748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b376b-9239-4975-b4b7-b72f858643c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:59.788823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b376b-9239-4975-b4b7-b72f858643c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.789290 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b376b-9239-4975-b4b7-b72f858643c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.789751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b376b-9239-4975-b4b7-b72f858643c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.791141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b376b-9239-4975-b4b7-b72f858643c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:59.794873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b376b-9239-4975-b4b7-b72f858643c6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:59.813455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1205/2417] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:45:59.815582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:45:59.828793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b31fe-15b3-4d9c-9b28-f243eb4f33d6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:45:59.834719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:45:59.836679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.836679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.836679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] PATH_INFO: `/v3/projects/511af3c2525246df8db1878465be924d/users/f308598076f44cb9a698b21bf1fd520f/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.856789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=511af3c2525246df8db1878465be924d, user_id=f308598076f44cb9a698b21bf1fd520f, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:45:59.861252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:45:59.895347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1213/2418] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:45:59 2022] PUT /identity/v3/projects/511af3c2525246df8db1878465be924d/users/f308598076f44cb9a698b21bf1fd520f/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:45:59.908358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016400-f1ef-4389-b24b-aa6e81c130f7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:45:59.908669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016400-f1ef-4389-b24b-aa6e81c130f7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:45:59.908937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43016400-f1ef-4389-b24b-aa6e81c130f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:45:59.948420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-43016400-f1ef-4389-b24b-aa6e81c130f7 None None] MFA Rules not processed for user `f308598076f44cb9a698b21bf1fd520f`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:45:59.950170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-43016400-f1ef-4389-b24b-aa6e81c130f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:00.020560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:00.069826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1206/2419] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:45:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:00.086907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:00.090359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:00.188613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e937b-8a7f-45a8-8136-efe945532fe0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:00.195300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfef8bf-7f9e-45fe-b6b2-f46d50f451f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f3b6ba74fb49a280735bb8c867d26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:00.196974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfef8bf-7f9e-45fe-b6b2-f46d50f451f9 service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:00.197439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfef8bf-7f9e-45fe-b6b2-f46d50f451f9 service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:00.198014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfef8bf-7f9e-45fe-b6b2-f46d50f451f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:00.212731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfef8bf-7f9e-45fe-b6b2-f46d50f451f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:00.215096 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfef8bf-7f9e-45fe-b6b2-f46d50f451f9 service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:00.230455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1214/2420] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:46:10.063023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=9 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:10.222293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:24.789739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43016400-f1ef-4389-b24b-aa6e81c130f7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:24.878344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43016400-f1ef-4389-b24b-aa6e81c130f7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:24.883764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:24.884796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:24.885122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:24.885483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] PATH_INFO: `/v3/users/f308598076f44cb9a698b21bf1fd520f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:24.886824 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f308598076f44cb9a698b21bf1fd520f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:24.888842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:24.947962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f308598076f44cb9a698b21bf1fd520f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:24.955422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f308598076f44cb9a698b21bf1fd520f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:24.964523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f308598076f44cb9a698b21bf1fd520f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:24.965157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f308598076f44cb9a698b21bf1fd520f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:24.978004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f308598076f44cb9a698b21bf1fd520f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:24.985070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1207/2421] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:46:24 2022] DELETE /identity/v3/users/f308598076f44cb9a698b21bf1fd520f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:46:24.990213 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:24.991010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:24.991010 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.018489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:25.020501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:25.096990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1215/2422] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:25.107995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:25.110129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:25.191591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73368cf-7ab5-4acc-8790-2eeb8aa60a33 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:25.195406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032715e-bb96-41aa-9ea9-f7e09872ecc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:25.196142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032715e-bb96-41aa-9ea9-f7e09872ecc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.196428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032715e-bb96-41aa-9ea9-f7e09872ecc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.197245 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032715e-bb96-41aa-9ea9-f7e09872ecc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.207781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032715e-bb96-41aa-9ea9-f7e09872ecc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:25.209324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032715e-bb96-41aa-9ea9-f7e09872ecc7 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:25.220742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1208/2423] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:46:25.612620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:25.614988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:25.638262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8550a-41fb-4b86-a3c5-f34a3b3a2dfc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:25.643579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:25.644493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.644862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.645138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] PATH_INFO: `/v3/projects/511af3c2525246df8db1878465be924d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.646465 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] RBAC: Authorizing `identity:delete_project(project_id=511af3c2525246df8db1878465be924d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:25.648378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:25.721356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '511af3c2525246df8db1878465be924d', 'request_id': 'req-6f01404c-c904-4d24-b87a-c820297f859f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:25.732914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '511af3c2525246df8db1878465be924d', 'request_id': 'req-6f01404c-c904-4d24-b87a-c820297f859f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:25.739969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '511af3c2525246df8db1878465be924d', 'request_id': 'req-6f01404c-c904-4d24-b87a-c820297f859f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:25.752520 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] The token cache is being invalidate because project 511af3c2525246df8db1878465be924d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:46:25.752811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:25.753478 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f01404c-c904-4d24-b87a-c820297f859f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:25.756337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1216/2424] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:46:25 2022] DELETE /identity/v3/projects/511af3c2525246df8db1878465be924d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:25.763947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a855fc4-8369-40c5-90e5-39900be33c0e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.764418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a855fc4-8369-40c5-90e5-39900be33c0e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.764894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a855fc4-8369-40c5-90e5-39900be33c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.790348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9a855fc4-8369-40c5-90e5-39900be33c0e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:25.791600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9a855fc4-8369-40c5-90e5-39900be33c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:25.814242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:25.866317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1209/2425] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:25.874880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee3b1a-08ec-4960-a851-96be19d46793 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.875166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee3b1a-08ec-4960-a851-96be19d46793 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.875440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee3b1a-08ec-4960-a851-96be19d46793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.897782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-88ee3b1a-08ec-4960-a851-96be19d46793 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:25.899253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-88ee3b1a-08ec-4960-a851-96be19d46793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:25.908138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1217/2426] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:46:25.912738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a855fc4-8369-40c5-90e5-39900be33c0e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:25.928093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a855fc4-8369-40c5-90e5-39900be33c0e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:25.933622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:25.934365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.934652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.934931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.935712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:46:25.936025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:25.937677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:25.951769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1210/2427] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:46:25.956476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee3b1a-08ec-4960-a851-96be19d46793 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:25.967188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee3b1a-08ec-4960-a851-96be19d46793 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:25.972586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:25.973476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:25.973942 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:25.974329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:25.975134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:25.977173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.008313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1218/2428] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:25 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:46:26.011283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.022437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57822b00-26e8-491e-9cef-4cb4b952a30f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.102531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.104191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.104657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.105125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.106289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.109609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.142775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1211/2429] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:26 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:26.147769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.166449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda797-8c0f-4652-a1b9-c1f8295bcf66 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.172647 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.173815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.174364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.174854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.176191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.178559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.193551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1219/2430] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:46:26.198674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.210252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b6424-6737-4aa6-9b02-8c088fe50422 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.215531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875795c-5c9d-4c10-ae97-bb5968379d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.216236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875795c-5c9d-4c10-ae97-bb5968379d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.216519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875795c-5c9d-4c10-ae97-bb5968379d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.216794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875795c-5c9d-4c10-ae97-bb5968379d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.217564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875795c-5c9d-4c10-ae97-bb5968379d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.220214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875795c-5c9d-4c10-ae97-bb5968379d4d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.239668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1212/2431] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:26.245683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.262502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13245cc9-0f70-45ca-8fec-4ce88e785f15 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.270338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.271657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.272203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.272691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] PATH_INFO: `/v3/projects/ea428606135542eea8d80f43fd416a3e/users/2ac5a102fecf4d4e8f91139efdb7a6ab/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.300019 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea428606135542eea8d80f43fd416a3e, user_id=2ac5a102fecf4d4e8f91139efdb7a6ab, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.303037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.320270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1220/2432] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:26 2022] PUT /identity/v3/projects/ea428606135542eea8d80f43fd416a3e/users/2ac5a102fecf4d4e8f91139efdb7a6ab/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:26.331405 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc597-449d-4372-86dc-90fea26b288c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.331800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc597-449d-4372-86dc-90fea26b288c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.332083 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc597-449d-4372-86dc-90fea26b288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.374894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-720dc597-449d-4372-86dc-90fea26b288c None None] MFA Rules not processed for user `2ac5a102fecf4d4e8f91139efdb7a6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:26.376389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-720dc597-449d-4372-86dc-90fea26b288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:26.453313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1213/2433] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:46:26.458448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.486498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:26.537601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c06c1a-e218-497d-8fbe-98e5727fd569 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.543414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.544100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.544389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.544668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.545387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.547354 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.580308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1221/2434] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:26 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:26.586725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dc597-449d-4372-86dc-90fea26b288c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.607423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dc597-449d-4372-86dc-90fea26b288c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.630881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:26.697984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.698741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.699077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.699361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.700077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.702062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.727067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1214/2435] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:26 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:26.731919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.742434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077a628-60c4-4641-9762-4b4d20066e1b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.749497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.750532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.750955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.751376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.752586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.754943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.774240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1222/2436] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:46:26.779211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.793515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43528e4-1353-4d26-985f-8d65afc7db4d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.799637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43d2f6-c5e0-4834-be09-d922174d795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.800495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43d2f6-c5e0-4834-be09-d922174d795a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.800850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43d2f6-c5e0-4834-be09-d922174d795a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.801163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43d2f6-c5e0-4834-be09-d922174d795a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.802100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43d2f6-c5e0-4834-be09-d922174d795a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.804459 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43d2f6-c5e0-4834-be09-d922174d795a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.817880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1215/2437] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:46:26.822911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:26.841908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22628eb-7dff-4e10-953f-d7f0ceb9c94f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:26.848261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:26.849413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.849599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.849962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] PATH_INFO: `/v3/projects/614a7a6d08c343d787a4fbe5501d330d/users/b3e3e34f353c4858bce69b8bba69b89a/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.867795 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=614a7a6d08c343d787a4fbe5501d330d, user_id=b3e3e34f353c4858bce69b8bba69b89a, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:26.870612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:26.887965 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1223/2438] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:26 2022] PUT /identity/v3/projects/614a7a6d08c343d787a4fbe5501d330d/users/b3e3e34f353c4858bce69b8bba69b89a/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:26.906947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e35315-9734-423a-b91f-be43d8c3dfc5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:26.907241 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e35315-9734-423a-b91f-be43d8c3dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:26.907524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e35315-9734-423a-b91f-be43d8c3dfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:26.940345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f9e35315-9734-423a-b91f-be43d8c3dfc5 None None] MFA Rules not processed for user `b3e3e34f353c4858bce69b8bba69b89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:26.941601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f9e35315-9734-423a-b91f-be43d8c3dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:27.030670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1216/2439] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:27.036664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:27.113357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2aa36-643b-47cd-a45e-7534907e20bb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:27.119037 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:27.119687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.119971 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.120246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:27.120980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:27.122980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:27.155289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1224/2440] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:27 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:27.160035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e35315-9734-423a-b91f-be43d8c3dfc5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:27.178772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e35315-9734-423a-b91f-be43d8c3dfc5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:27.275376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:27.276128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.276452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.276725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:27.277442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:27.279862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:27.311253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1217/2441] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:27 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:27.316603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:27.334263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74915d-76ba-435e-aad1-e620871df66a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:27.342333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:27.343443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.343921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.344378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:27.345403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:27.347949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:27.361015 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1225/2442] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:27.365948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:27.379850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a88b7-3d9c-4695-ae4b-d573d324ec55 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:27.388712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:27.390381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.390926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.391432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] PATH_INFO: `/v3/projects/389517ec78764be2b2a9be4ee2e5387f/users/6992895dc52f4796a3b63f009411ae99/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:27.415115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:27.424462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=389517ec78764be2b2a9be4ee2e5387f, user_id=6992895dc52f4796a3b63f009411ae99, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:27.429484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:27.463261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1218/2443] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:27 2022] PUT /identity/v3/projects/389517ec78764be2b2a9be4ee2e5387f/users/6992895dc52f4796a3b63f009411ae99/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:46:27.468535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:27.481716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a0c6c-838e-4e19-932b-e2260d39a50d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:27.549931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:27.574232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a65b0-bc78-4375-8ee8-ed05093c0ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:27.575031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a65b0-bc78-4375-8ee8-ed05093c0ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.575366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a65b0-bc78-4375-8ee8-ed05093c0ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.575678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a65b0-bc78-4375-8ee8-ed05093c0ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:27.576605 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a65b0-bc78-4375-8ee8-ed05093c0ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:27.578333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a65b0-bc78-4375-8ee8-ed05093c0ff8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:27.594768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1226/2444] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:46:27.601818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:27.618135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f0b85-bdce-47e4-a4cc-3f064b7d7e85 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:27.626705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:27.628051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.628481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.628872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] PATH_INFO: `/v3/projects/389517ec78764be2b2a9be4ee2e5387f/users/6992895dc52f4796a3b63f009411ae99/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:27.631159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=389517ec78764be2b2a9be4ee2e5387f, user_id=6992895dc52f4796a3b63f009411ae99, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:27.633852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:27.650435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1219/2445] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:27 2022] PUT /identity/v3/projects/389517ec78764be2b2a9be4ee2e5387f/users/6992895dc52f4796a3b63f009411ae99/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:27.667472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.667924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.668246 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:27.715259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] MFA Rules not processed for user `6992895dc52f4796a3b63f009411ae99`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:27.717343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:27.870164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1227/2446] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:27.890063 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:27.893355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:27.988492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c6fb0-a289-4d35-8030-3e21213cce0a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:27.992464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:27.994319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:27.994319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:27.994319 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:28.077813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:28.080873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:28.098329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1220/2447] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:29.786929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:29.786929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:29.803184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e547-a9a4-4f93-a83e-1466db6d6bc8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:29.808140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381b3dd-6208-4b39-801c-63e6ff6e38d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:29.808910 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b3dd-6208-4b39-801c-63e6ff6e38d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:29.809292 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b3dd-6208-4b39-801c-63e6ff6e38d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:29.809614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381b3dd-6208-4b39-801c-63e6ff6e38d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:29.882704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b3dd-6208-4b39-801c-63e6ff6e38d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:29.886064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381b3dd-6208-4b39-801c-63e6ff6e38d1 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:29.901359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1228/2448] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:30.615166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:30.638733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270890-9bca-4729-a846-d280fe426b8a service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:30.644781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:30.645560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:30.645991 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:30.646367 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:30.657260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:30.658767 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:30.669285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1221/2449] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:46:34.428097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381fa4-10d1-447b-9333-a652d34b061c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.430329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381fa4-10d1-447b-9333-a652d34b061c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.430711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60381fa4-10d1-447b-9333-a652d34b061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.459233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-60381fa4-10d1-447b-9333-a652d34b061c None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:34.461083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-60381fa4-10d1-447b-9333-a652d34b061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:34.542073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1229/2450] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:46:34.600505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:34.602105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:34.739121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321543b5-ce11-414d-a844-437aba6cb2a6 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:34.756919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381fa4-10d1-447b-9333-a652d34b061c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:34.771687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5440b57-ad51-4bf7-ab72-ad882803957b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:34.772032 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:34.773091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440b57-ad51-4bf7-ab72-ad882803957b service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.773504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440b57-ad51-4bf7-ab72-ad882803957b service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.773957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440b57-ad51-4bf7-ab72-ad882803957b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.788637 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5440b57-ad51-4bf7-ab72-ad882803957b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:34.791430 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5440b57-ad51-4bf7-ab72-ad882803957b service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.807396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1222/2451] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:46:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:46:34.854907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60381fa4-10d1-447b-9333-a652d34b061c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:34.863555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:34.863555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:34.863953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:34.864552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] PATH_INFO: `/v3/users/2ac5a102fecf4d4e8f91139efdb7a6ab` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:34.867418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac5a102fecf4d4e8f91139efdb7a6ab)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:34.872142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:34.954619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac5a102fecf4d4e8f91139efdb7a6ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:34.973290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac5a102fecf4d4e8f91139efdb7a6ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:34.994256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac5a102fecf4d4e8f91139efdb7a6ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:34.995662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac5a102fecf4d4e8f91139efdb7a6ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:35.026118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac5a102fecf4d4e8f91139efdb7a6ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:35.040923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1230/2452] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:46:34 2022] DELETE /identity/v3/users/2ac5a102fecf4d4e8f91139efdb7a6ab => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:35.057977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:35.058535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:35.059670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:35.107504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:35.108920 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:35.116379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:35.275185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1223/2453] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:35.292003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:35.294262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:35.454559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d6345-95b2-478a-918a-fb39cc0b4322 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:35.461226 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:35.462977 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:35.463451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:35.470051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:35.508609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:35.512654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:35.562149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1231/2454] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:36.086857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:36.090574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:36.128480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a690ba-167f-481e-b86b-415d1dd31df4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:36.139512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:36.141345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:36.142094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:36.142936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] PATH_INFO: `/v3/projects/ea428606135542eea8d80f43fd416a3e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:36.145026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea428606135542eea8d80f43fd416a3e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:36.148582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:36.275557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea428606135542eea8d80f43fd416a3e', 'request_id': 'req-b53186ac-3455-4594-968e-1d6c04ec0dca'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.291308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea428606135542eea8d80f43fd416a3e', 'request_id': 'req-b53186ac-3455-4594-968e-1d6c04ec0dca'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.304345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea428606135542eea8d80f43fd416a3e', 'request_id': 'req-b53186ac-3455-4594-968e-1d6c04ec0dca'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.326215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] The token cache is being invalidate because project ea428606135542eea8d80f43fd416a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:46:36.326714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.328333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.332077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1224/2455] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:46:36 2022] DELETE /identity/v3/projects/ea428606135542eea8d80f43fd416a3e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:36.338801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:36.357071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e7ae-3f22-47e4-88b5-98bd74194da1 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:36.427558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:36.511127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:36.513792 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:36.514517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:36.515034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] PATH_INFO: `/v3/users/b3e3e34f353c4858bce69b8bba69b89a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:36.517303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e3e34f353c4858bce69b8bba69b89a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:36.521172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:36.706376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e3e34f353c4858bce69b8bba69b89a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.723290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e3e34f353c4858bce69b8bba69b89a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.747405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e3e34f353c4858bce69b8bba69b89a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.754108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e3e34f353c4858bce69b8bba69b89a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.788195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e3e34f353c4858bce69b8bba69b89a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:36.809105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1232/2456] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:46:36 2022] DELETE /identity/v3/users/b3e3e34f353c4858bce69b8bba69b89a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:37.394199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:37.397289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:37.448328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53186ac-3455-4594-968e-1d6c04ec0dca admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:37.568619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:37.570455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:37.571355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:37.571940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] PATH_INFO: `/v3/projects/614a7a6d08c343d787a4fbe5501d330d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:37.575554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=614a7a6d08c343d787a4fbe5501d330d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:37.578037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:37.677351 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:37.703073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614a7a6d08c343d787a4fbe5501d330d', 'request_id': 'req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:37.717455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614a7a6d08c343d787a4fbe5501d330d', 'request_id': 'req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:37.735402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614a7a6d08c343d787a4fbe5501d330d', 'request_id': 'req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:37.764999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] The token cache is being invalidate because project 614a7a6d08c343d787a4fbe5501d330d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:46:37.765311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:37.766941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:37.771399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1225/2457] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:46:37 2022] DELETE /identity/v3/projects/614a7a6d08c343d787a4fbe5501d330d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:46:37.777332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:37.779797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:37.813161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416be817-40c4-4c96-a575-fa20c6316016 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:37.957254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:37.958112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:37.958539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:37.958982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] PATH_INFO: `/v3/users/6992895dc52f4796a3b63f009411ae99` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:37.960278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6992895dc52f4796a3b63f009411ae99)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:37.962874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:38.063662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6992895dc52f4796a3b63f009411ae99'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.068241 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:38.081585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6992895dc52f4796a3b63f009411ae99'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.122070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6992895dc52f4796a3b63f009411ae99'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.122828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6992895dc52f4796a3b63f009411ae99'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.152536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6becd74f-1c99-4305-b137-49d2eaa11b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6992895dc52f4796a3b63f009411ae99'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.161899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1233/2458] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:46:37 2022] DELETE /identity/v3/users/6992895dc52f4796a3b63f009411ae99 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:38.623794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:38.642752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:38.682539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea90363-8534-4a66-9ea8-8aa6b9f18bbe admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:38.810286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:38.811418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:38.811817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:38.812189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] PATH_INFO: `/v3/projects/389517ec78764be2b2a9be4ee2e5387f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:38.813594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] RBAC: Authorizing `identity:delete_project(project_id=389517ec78764be2b2a9be4ee2e5387f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:38.815898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:38.947440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '389517ec78764be2b2a9be4ee2e5387f', 'request_id': 'req-05dada9b-d689-4174-aa2c-3a544e2e974e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.957760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '389517ec78764be2b2a9be4ee2e5387f', 'request_id': 'req-05dada9b-d689-4174-aa2c-3a544e2e974e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.970185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '389517ec78764be2b2a9be4ee2e5387f', 'request_id': 'req-05dada9b-d689-4174-aa2c-3a544e2e974e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.992948 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] The token cache is being invalidate because project 389517ec78764be2b2a9be4ee2e5387f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:46:38.994092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.995225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05dada9b-d689-4174-aa2c-3a544e2e974e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:46:38.998101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1226/2459] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:46:38 2022] DELETE /identity/v3/projects/389517ec78764be2b2a9be4ee2e5387f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:39.005413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7aaab-ca73-45f5-aecb-6bd0faf04cdf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.006233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7aaab-ca73-45f5-aecb-6bd0faf04cdf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.006467 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7aaab-ca73-45f5-aecb-6bd0faf04cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.036189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-48d7aaab-ca73-45f5-aecb-6bd0faf04cdf None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:39.037819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-48d7aaab-ca73-45f5-aecb-6bd0faf04cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:39.141203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1234/2460] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:39.153759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0596bd9-0a8d-460d-9743-842e6597940b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.154301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0596bd9-0a8d-460d-9743-842e6597940b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.154795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0596bd9-0a8d-460d-9743-842e6597940b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.188235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d0596bd9-0a8d-460d-9743-842e6597940b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:39.190502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0596bd9-0a8d-460d-9743-842e6597940b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:39.203830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1227/2461] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:39.210420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7aaab-ca73-45f5-aecb-6bd0faf04cdf None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:39.231130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7aaab-ca73-45f5-aecb-6bd0faf04cdf None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:39.251071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:39.252169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.252729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.253585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.254609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:46:39.255409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:39.258558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:39.291896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1235/2462] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:46:39.294891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0596bd9-0a8d-460d-9743-842e6597940b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:39.314501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0596bd9-0a8d-460d-9743-842e6597940b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:39.322210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:39.322898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.323182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.323466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.324200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:39.326161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:39.372864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1228/2463] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:46:39 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:39.377456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:39.403545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722040-885a-424f-9936-93479d1eaf8d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:39.569569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:39.570756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.571284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.571773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.572831 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:39.575495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:39.606523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1236/2464] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:46:39 2022] POST /identity/v3/users => generated 284 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:46:39.627199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:39.658080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61af7c-3bc8-4257-89f6-66fe6b42cb45 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:39.687540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:39.689740 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.690479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.690855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.692054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:39.694026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:39.714425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1229/2465] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:39.719131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:39.736904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a091e80-6a9b-4aa9-bac1-d91a038cece4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:39.745528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807966fd-9008-45d1-a3d8-b9e182af9617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:39.746756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807966fd-9008-45d1-a3d8-b9e182af9617 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.747265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807966fd-9008-45d1-a3d8-b9e182af9617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.747896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807966fd-9008-45d1-a3d8-b9e182af9617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.749310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807966fd-9008-45d1-a3d8-b9e182af9617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:39.753612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807966fd-9008-45d1-a3d8-b9e182af9617 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:39.768000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1237/2466] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:46:39.774638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:39.798861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e87d26-d469-4265-8550-a13838873d74 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:39.813296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:39.814842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.815303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.815902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] PATH_INFO: `/v3/projects/9696da4e904f4a99881a6d15fcde3cb0/users/b70011fa2bda424f9b680adb82a9394d/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.839077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:39.844872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9696da4e904f4a99881a6d15fcde3cb0, user_id=b70011fa2bda424f9b680adb82a9394d, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:39.848518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:39.881550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1230/2467] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:46:39 2022] PUT /identity/v3/projects/9696da4e904f4a99881a6d15fcde3cb0/users/b70011fa2bda424f9b680adb82a9394d/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:46:39.896658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:39.897121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:39.897621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:39.947789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] MFA Rules not processed for user `b70011fa2bda424f9b680adb82a9394d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:39.954291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:40.000762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:40.076273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1238/2468] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:46:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:40.098346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:40.101892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:40.202554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6faf9a-1806-4950-b17a-53b2f7cf20ea admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:40.208867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414d278-9132-4f68-8a6d-67a984d5703c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f3b6ba74fb49a280735bb8c867d26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:40.208867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414d278-9132-4f68-8a6d-67a984d5703c service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:40.208867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414d278-9132-4f68-8a6d-67a984d5703c service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:40.208867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414d278-9132-4f68-8a6d-67a984d5703c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:40.226542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414d278-9132-4f68-8a6d-67a984d5703c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:40.229462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414d278-9132-4f68-8a6d-67a984d5703c service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:40.245411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1231/2469] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:46:50.071126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=16 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:50.238485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:46:55.796504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:55.801666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:55.930281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1274ca5-2f31-4952-b12a-f41595a19279 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:55.937729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:55.938670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:55.939162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:55.939663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:55.942436 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:55.964860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:55.968014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:55.988143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1239/2470] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:46:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:46:58.866467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242893b-cd03-49ca-90f8-ecbf093d7c3a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:58.867046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242893b-cd03-49ca-90f8-ecbf093d7c3a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:58.867841 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242893b-cd03-49ca-90f8-ecbf093d7c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:58.894508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7242893b-cd03-49ca-90f8-ecbf093d7c3a None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:46:58.895751 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7242893b-cd03-49ca-90f8-ecbf093d7c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:58.999330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1232/2471] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:46:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:46:59.016610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:46:59.018876 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:46:59.165192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6caaf7-5747-4a51-887d-f89811ed120d service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:46:59.172693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:46:59.174441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:46:59.175043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:46:59.175381 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:46:59.186245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:46:59.189385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:46:59.207370 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1240/2472] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:46:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:47:08.988872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=25 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:09.204913 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=64 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:16.112111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e5303-66ce-4fba-96d2-aa421f08933c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:16.112927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e5303-66ce-4fba-96d2-aa421f08933c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:16.113270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e5303-66ce-4fba-96d2-aa421f08933c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:16.140098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-336e5303-66ce-4fba-96d2-aa421f08933c None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:16.141489 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-336e5303-66ce-4fba-96d2-aa421f08933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:16.150761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1233/2473] 10.222.0.25 () {58 vars in 967 bytes} [Fri Jun 17 11:47:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:16.168420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:16.186730 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeee7a3-ad4d-4adb-bc49-53ce970c48ae service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:16.193882 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:16.195361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:16.195641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:16.196140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:16.207008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:16.208596 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:16.219723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1241/2474] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:47:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:47:18.466916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac87bf9-5e73-4bce-ad50-8c31271d3e2b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:18.467480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac87bf9-5e73-4bce-ad50-8c31271d3e2b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:18.467726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac87bf9-5e73-4bce-ad50-8c31271d3e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:18.495770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6ac87bf9-5e73-4bce-ad50-8c31271d3e2b None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:18.498239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6ac87bf9-5e73-4bce-ad50-8c31271d3e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:18.515300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1234/2475] 10.222.0.25 () {58 vars in 967 bytes} [Fri Jun 17 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:18.529036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:18.539831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac87bf9-5e73-4bce-ad50-8c31271d3e2b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:18.557355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac87bf9-5e73-4bce-ad50-8c31271d3e2b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:18.563506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76a644-f49e-48c1-a759-89eac6555cbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:18.564183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a644-f49e-48c1-a759-89eac6555cbc service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:18.565799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a644-f49e-48c1-a759-89eac6555cbc service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:18.566333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76a644-f49e-48c1-a759-89eac6555cbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:18.578769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76a644-f49e-48c1-a759-89eac6555cbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:18.580729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76a644-f49e-48c1-a759-89eac6555cbc service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:18.600416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1235/2476] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:47:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:47:18.704337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3853b4-4046-4900-8d4f-a7402cbf5fc4 service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:18.721880 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:18.722802 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:18.723240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:18.723522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] PATH_INFO: `/v3/users/b70011fa2bda424f9b680adb82a9394d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:18.725773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70011fa2bda424f9b680adb82a9394d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:18.738070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:18.836660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70011fa2bda424f9b680adb82a9394d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.854140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70011fa2bda424f9b680adb82a9394d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.865199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70011fa2bda424f9b680adb82a9394d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.867072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70011fa2bda424f9b680adb82a9394d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.889655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70011fa2bda424f9b680adb82a9394d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:18.913918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1242/2477] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:18 2022] DELETE /identity/v3/users/b70011fa2bda424f9b680adb82a9394d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:18.918815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:18.919013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:18.919267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:18.941680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:18.943399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:19.065204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1236/2478] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:19.082811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:19.086770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:19.269627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c1f55-1c3a-4554-93d9-6433fe1f054e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:19.283805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:19.284811 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:19.285242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:19.285668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:19.324866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:19.344709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:19.369144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1243/2479] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:47:20.329685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:20.333114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:20.407190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0375591-739b-4eee-9f3b-32a195ac3ece None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:20.435383 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:20.438326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:20.439847 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:20.441693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] PATH_INFO: `/v3/projects/9696da4e904f4a99881a6d15fcde3cb0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:20.445200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9696da4e904f4a99881a6d15fcde3cb0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:20.449995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:20.635825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9696da4e904f4a99881a6d15fcde3cb0', 'request_id': 'req-3e1e2648-648d-48e2-86f3-0cc2235333dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.646177 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9696da4e904f4a99881a6d15fcde3cb0', 'request_id': 'req-3e1e2648-648d-48e2-86f3-0cc2235333dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.656919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9696da4e904f4a99881a6d15fcde3cb0', 'request_id': 'req-3e1e2648-648d-48e2-86f3-0cc2235333dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.714343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] The token cache is being invalidate because project 9696da4e904f4a99881a6d15fcde3cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:20.714688 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.715446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:20.722536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1237/2480] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:20 2022] DELETE /identity/v3/projects/9696da4e904f4a99881a6d15fcde3cb0 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:26.788123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:26.789604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:26.811616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60868e1d-39f7-49f4-bdd6-a9496b9d789c service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:26.821503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:26.926491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:26.927380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:26.927919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:26.928755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:26.938700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:27.086909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:27.088416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:27.109756 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1244/2481] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:28.015820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:28.018075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:28.032535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e2648-648d-48e2-86f3-0cc2235333dd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:28.047159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:28.049189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:28.049681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:28.050218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] PATH_INFO: `/v3/users/0f1328cad6ed4e1b8f446894f81f3fb4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:28.054007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1328cad6ed4e1b8f446894f81f3fb4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:28.058326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:28.151440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1328cad6ed4e1b8f446894f81f3fb4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.161887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:28.176021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1328cad6ed4e1b8f446894f81f3fb4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.193724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1328cad6ed4e1b8f446894f81f3fb4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.193724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1328cad6ed4e1b8f446894f81f3fb4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.214399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1328cad6ed4e1b8f446894f81f3fb4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.235296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1238/2482] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:28 2022] DELETE /identity/v3/users/0f1328cad6ed4e1b8f446894f81f3fb4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:28.473320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:28.500518 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa87ce-e76a-4542-bc56-ff103c4367a9 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:28.638448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:28.639350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:28.639922 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:28.641018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] PATH_INFO: `/v3/projects/21cd7e2263234026bd12c574c137e311` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:28.643258 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21cd7e2263234026bd12c574c137e311)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:28.645403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:28.710444 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21cd7e2263234026bd12c574c137e311', 'request_id': 'req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.722801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21cd7e2263234026bd12c574c137e311', 'request_id': 'req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.728123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:28.731350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21cd7e2263234026bd12c574c137e311', 'request_id': 'req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.749260 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] The token cache is being invalidate because project 21cd7e2263234026bd12c574c137e311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:28.749669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.750585 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.752389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1245/2483] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:28 2022] DELETE /identity/v3/projects/21cd7e2263234026bd12c574c137e311 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:28.757978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:28.759699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:28.793794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c56c6e-7a52-4419-8a71-de0aac4ccac5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:28.872588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:28.873683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:28.873999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:28.874279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] PATH_INFO: `/v3/users/f64877c00a7e4b5483a6940c1c93369d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:28.875598 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64877c00a7e4b5483a6940c1c93369d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:28.878024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:28.946313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64877c00a7e4b5483a6940c1c93369d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.956661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64877c00a7e4b5483a6940c1c93369d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.967424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64877c00a7e4b5483a6940c1c93369d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.968363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64877c00a7e4b5483a6940c1c93369d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:28.992916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64877c00a7e4b5483a6940c1c93369d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:29.011788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1239/2484] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:28 2022] DELETE /identity/v3/users/f64877c00a7e4b5483a6940c1c93369d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:29.362431 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:29.364528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:29.384034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1ba5b-578d-4bdb-9b94-3094a92a7864 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:29.484029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:29.487924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:29.487924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:29.487924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] PATH_INFO: `/v3/projects/83362f3cee7b46e1a1e44e630d2866a2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:29.487924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83362f3cee7b46e1a1e44e630d2866a2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:29.491892 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:29.553758 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83362f3cee7b46e1a1e44e630d2866a2', 'request_id': 'req-0b3624d8-f349-4513-854a-c366b0fbb9a9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:29.562953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83362f3cee7b46e1a1e44e630d2866a2', 'request_id': 'req-0b3624d8-f349-4513-854a-c366b0fbb9a9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:29.572174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83362f3cee7b46e1a1e44e630d2866a2', 'request_id': 'req-0b3624d8-f349-4513-854a-c366b0fbb9a9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:29.586594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] The token cache is being invalidate because project 83362f3cee7b46e1a1e44e630d2866a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:29.587092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:29.588222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:29.589833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1246/2485] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:29 2022] DELETE /identity/v3/projects/83362f3cee7b46e1a1e44e630d2866a2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:32.111542 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:32.114310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:32.146124 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3e4c1-87dc-4a0c-8934-fb27b226c386 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:32.231668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:32.255185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:32.256128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:32.256425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:32.256706 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] PATH_INFO: `/v3/users/3a0f25c7ce194ddbb21d1c8b97d03c8c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:32.258000 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0f25c7ce194ddbb21d1c8b97d03c8c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:32.260064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:32.331811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0f25c7ce194ddbb21d1c8b97d03c8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.342285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0f25c7ce194ddbb21d1c8b97d03c8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.352837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0f25c7ce194ddbb21d1c8b97d03c8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.353480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0f25c7ce194ddbb21d1c8b97d03c8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.366912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af3279f6-9b92-40cc-b6b7-87ff0ca263f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0f25c7ce194ddbb21d1c8b97d03c8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.373872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1240/2486] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:32 2022] DELETE /identity/v3/users/3a0f25c7ce194ddbb21d1c8b97d03c8c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:32.587658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:32.589617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:32.611579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3624d8-f349-4513-854a-c366b0fbb9a9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:32.702317 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:32.703192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:32.703469 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:32.703744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] PATH_INFO: `/v3/projects/fad50b8414f246428a34e0465810ab97` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:32.704968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad50b8414f246428a34e0465810ab97)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:32.706924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:32.736261 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:32.774899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad50b8414f246428a34e0465810ab97', 'request_id': 'req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.784229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad50b8414f246428a34e0465810ab97', 'request_id': 'req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.793116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad50b8414f246428a34e0465810ab97', 'request_id': 'req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.806116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] The token cache is being invalidate because project fad50b8414f246428a34e0465810ab97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:32.806404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.807066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2f5b868d-5cd3-4fe9-b8a6-6db22a0aef77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:32.808775 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1247/2487] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:32 2022] DELETE /identity/v3/projects/fad50b8414f246428a34e0465810ab97 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:32.815945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99522-ceda-4e43-8f75-4a9f2174b496 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:32.816231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99522-ceda-4e43-8f75-4a9f2174b496 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:32.816507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99522-ceda-4e43-8f75-4a9f2174b496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:32.840025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-23c99522-ceda-4e43-8f75-4a9f2174b496 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:32.841315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-23c99522-ceda-4e43-8f75-4a9f2174b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:32.943190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1241/2488] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:32.947909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea48056-ef2b-4982-9a84-32e59bdfa73d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:32.948204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea48056-ef2b-4982-9a84-32e59bdfa73d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:32.948495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea48056-ef2b-4982-9a84-32e59bdfa73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:32.976898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3ea48056-ef2b-4982-9a84-32e59bdfa73d None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:32.978163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3ea48056-ef2b-4982-9a84-32e59bdfa73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:32.986888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1248/2489] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:32.991329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c99522-ceda-4e43-8f75-4a9f2174b496 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.007664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c99522-ceda-4e43-8f75-4a9f2174b496 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.012465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.013159 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.013443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.013718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.014498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:47:33.014813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:33.016465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:33.033966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1242/2490] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:47:33.036049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea48056-ef2b-4982-9a84-32e59bdfa73d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.049879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea48056-ef2b-4982-9a84-32e59bdfa73d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.051803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.052516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.053337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.053620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.054387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:33.056632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:33.095985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1249/2491] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:33 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:33.097492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.110076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b55ce-4990-4b2d-b49a-cc012d6e6778 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.184997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:33.199676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.200683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.201057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.201474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.202467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:33.204513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:33.226299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1243/2492] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:33 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:33.231084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.242787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4e52a-555e-4144-b4c1-967923739800 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.248405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.249383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.249764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.250154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.251243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:33.253402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:33.269451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1250/2493] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:33.275191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.286482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55de0-84f2-4413-9afa-e0f583d045f3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.292161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508682f2-38cb-4420-8bbb-b244453c27ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.292932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508682f2-38cb-4420-8bbb-b244453c27ad admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.293425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508682f2-38cb-4420-8bbb-b244453c27ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.294120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508682f2-38cb-4420-8bbb-b244453c27ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.294675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508682f2-38cb-4420-8bbb-b244453c27ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:33.296621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508682f2-38cb-4420-8bbb-b244453c27ad admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:33.309697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1244/2494] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:33.315229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.326364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43abe7a-92d9-4a26-b9f9-03ca836f68b3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.331240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.332138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.332427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.332710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] PATH_INFO: `/v3/projects/1b8bbe2704a748648ded1ad83976d903/users/8ae7bebba8f44afda8ea57208fb4470e/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.356904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8bbe2704a748648ded1ad83976d903, user_id=8ae7bebba8f44afda8ea57208fb4470e, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:33.362133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:33.380673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1251/2495] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:33 2022] PUT /identity/v3/projects/1b8bbe2704a748648ded1ad83976d903/users/8ae7bebba8f44afda8ea57208fb4470e/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:33.398504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.398966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.399473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.436657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] MFA Rules not processed for user `8ae7bebba8f44afda8ea57208fb4470e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:33.437884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:33.532184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1245/2496] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:33.547607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:33.549578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:33.639315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3f966-3b66-4fb2-a31d-0edd7b4acf60 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:33.645545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:33.646401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:33.646784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:33.647133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:33.657112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:33.658820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:33.669072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1252/2497] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:34.324337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:34.325918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:34.388332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba772696-6c5b-4b88-a912-d25edeb6caa3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:34.392095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494f239-8dcf-463f-879f-105299f21a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:34.392806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494f239-8dcf-463f-879f-105299f21a2c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:34.393226 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494f239-8dcf-463f-879f-105299f21a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:34.393626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494f239-8dcf-463f-879f-105299f21a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:34.395764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f494f239-8dcf-463f-879f-105299f21a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:34.467963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494f239-8dcf-463f-879f-105299f21a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:34.469554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f494f239-8dcf-463f-879f-105299f21a2c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:34.480029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1246/2498] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:36.471430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:36.492454 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf104aa-a2b6-4509-97eb-56e2fb1aa3a8 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:36.501089 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:36.502610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:36.503338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:36.504301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] PATH_INFO: `/v3/users/8ae7bebba8f44afda8ea57208fb4470e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:36.506789 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae7bebba8f44afda8ea57208fb4470e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:36.510772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:36.565560 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:36.606021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae7bebba8f44afda8ea57208fb4470e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:36.618507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae7bebba8f44afda8ea57208fb4470e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:36.633490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae7bebba8f44afda8ea57208fb4470e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:36.635862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae7bebba8f44afda8ea57208fb4470e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:36.664129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae7bebba8f44afda8ea57208fb4470e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:36.679112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1253/2499] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:36 2022] DELETE /identity/v3/users/8ae7bebba8f44afda8ea57208fb4470e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:47:36.684908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:36.685519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:36.685519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:36.703837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:36.705449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:36.739044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:36.814349 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1247/2500] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:36.825516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:36.827929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:36.933439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f493a-ed2e-4835-9ea7-85f6f1ecc6eb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:36.938321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b58d8c-bf83-433e-8c95-2b9ee6eb2808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:36.939244 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b58d8c-bf83-433e-8c95-2b9ee6eb2808 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:36.939663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b58d8c-bf83-433e-8c95-2b9ee6eb2808 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:36.939957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b58d8c-bf83-433e-8c95-2b9ee6eb2808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:36.951894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b58d8c-bf83-433e-8c95-2b9ee6eb2808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:36.953497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b58d8c-bf83-433e-8c95-2b9ee6eb2808 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:36.967276 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1254/2501] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:47:37.311792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:37.314374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:37.345137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eaab3c-bcf4-4c0f-8794-a6bae1f089dd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:37.353086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:37.354587 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:37.355119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:37.355632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] PATH_INFO: `/v3/projects/1b8bbe2704a748648ded1ad83976d903` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:37.357425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b8bbe2704a748648ded1ad83976d903)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:37.360838 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:37.444611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b8bbe2704a748648ded1ad83976d903', 'request_id': 'req-18ca2f72-95f5-448d-8319-044894bb56fe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:37.464531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b8bbe2704a748648ded1ad83976d903', 'request_id': 'req-18ca2f72-95f5-448d-8319-044894bb56fe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:37.472398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b8bbe2704a748648ded1ad83976d903', 'request_id': 'req-18ca2f72-95f5-448d-8319-044894bb56fe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:37.492819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] The token cache is being invalidate because project 1b8bbe2704a748648ded1ad83976d903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:37.493219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:37.493819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-18ca2f72-95f5-448d-8319-044894bb56fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:37.495416 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1248/2502] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:37 2022] DELETE /identity/v3/projects/1b8bbe2704a748648ded1ad83976d903 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:37.500807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42149-fe8b-4afd-a47d-aa45f23223f2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:37.500807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42149-fe8b-4afd-a47d-aa45f23223f2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:37.502296 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42149-fe8b-4afd-a47d-aa45f23223f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:37.521545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9df42149-fe8b-4afd-a47d-aa45f23223f2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:37.523358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9df42149-fe8b-4afd-a47d-aa45f23223f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:37.610728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1255/2503] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:37.619070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f2e24-62b8-468a-aba8-b9fdcca229aa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:37.619731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f2e24-62b8-468a-aba8-b9fdcca229aa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:37.628211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f2e24-62b8-468a-aba8-b9fdcca229aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:37.661036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7c1f2e24-62b8-468a-aba8-b9fdcca229aa None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:37.662369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7c1f2e24-62b8-468a-aba8-b9fdcca229aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:37.679517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1249/2504] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:37.681420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42149-fe8b-4afd-a47d-aa45f23223f2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:37.695727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42149-fe8b-4afd-a47d-aa45f23223f2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:37.705177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:37.705823 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:37.706146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:37.706430 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:37.707342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:47:37.707844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:37.709697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:37.728379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1256/2505] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:47:37.737814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f2e24-62b8-468a-aba8-b9fdcca229aa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:37.772244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f2e24-62b8-468a-aba8-b9fdcca229aa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:37.780841 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:37.782295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:37.782852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:37.783388 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:37.784623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:37.788201 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:37.826456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1250/2506] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:37 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:37.830711 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:37.851458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919e20f-4ba7-48c7-b14d-d58d116d8f7f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:37.926940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:37.987350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:37.988502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:37.989185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:37.989678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:37.991883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:37.995239 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:38.045905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1257/2507] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:37 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:47:38.049995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:38.064884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ed3d7-751f-4241-9653-dd42800f655e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:38.085580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:38.087205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.087947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.088451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.089422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:38.093072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:38.115466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1251/2508] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:47:38.119870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:38.154632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137382d-3912-43ff-9358-90a7823ffd04 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:38.164394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e764f40-7e7f-4b56-ae91-3e26ebeabaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:38.165288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e764f40-7e7f-4b56-ae91-3e26ebeabaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.165598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e764f40-7e7f-4b56-ae91-3e26ebeabaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.166152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e764f40-7e7f-4b56-ae91-3e26ebeabaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.167554 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e764f40-7e7f-4b56-ae91-3e26ebeabaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:38.170757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e764f40-7e7f-4b56-ae91-3e26ebeabaaa admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:38.214917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1258/2509] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:38.217032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:38.251081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d9e42-d483-435f-b8c4-b3259d4b7798 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:38.272674 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:38.274393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.274895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.275415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] PATH_INFO: `/v3/projects/dc35ed58a1614408b53949b6e2b10a8f/users/27a55bbc7d8b4a489f8aa35af867298e/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.301675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc35ed58a1614408b53949b6e2b10a8f, user_id=27a55bbc7d8b4a489f8aa35af867298e, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:38.306019 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:38.341554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1252/2510] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:38 2022] PUT /identity/v3/projects/dc35ed58a1614408b53949b6e2b10a8f/users/27a55bbc7d8b4a489f8aa35af867298e/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:38.355548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.355945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.356242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.411286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:38.415536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:38.420303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] MFA Rules not processed for user `27a55bbc7d8b4a489f8aa35af867298e`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:38.422184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:38.491361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:38.554486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1259/2511] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:38.555284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:38.557286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:38.574544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7f7d1-d8e3-46de-bde9-22eeb528f697 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:38.583568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:38.584750 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.585253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.585765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.589199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:38.610265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:38.613286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:38.632209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1253/2512] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:47:38.633590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:38.667033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbf615-cf6b-4082-873c-9a529e5ab660 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:38.672638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:38.673053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.673334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.673613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] PATH_INFO: `/v3/users/c2c930cfab47464fb784c98f58abbb2c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.675045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c930cfab47464fb784c98f58abbb2c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:38.677082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:38.747697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491c9fe-a276-46d9-8942-a264b625ef4b service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:38.750912 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:38.754402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:38.754710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c930cfab47464fb784c98f58abbb2c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:38.755387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.756143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.756614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.757530 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:38.761009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:38.764940 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c930cfab47464fb784c98f58abbb2c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:38.774582 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c930cfab47464fb784c98f58abbb2c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:38.775459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c930cfab47464fb784c98f58abbb2c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:38.799641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1254/2513] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:38 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:38.800263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c930cfab47464fb784c98f58abbb2c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:38.801531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:38.804636 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:38.810106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1260/2514] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:47:38 2022] DELETE /identity/v3/users/c2c930cfab47464fb784c98f58abbb2c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:38.912645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277699f-8065-47fe-b683-6206703ed310 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:38.983385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:38.993705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:38.994656 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:38.994943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:38.995219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:38.995947 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:38.998410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.034821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1255/2515] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:38 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:47:39.041574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.056254 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc05019-232d-450e-984a-38d28655ea32 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.059902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.062575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.062800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.063493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.063935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.064369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.065272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.067505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.078025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e8504-eecc-489e-afaf-9df025ca83be admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.083216 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.085024 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.085308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.085911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] PATH_INFO: `/v3/projects/03b64e74629c4d6b9c5ef82247509762` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.087333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b64e74629c4d6b9c5ef82247509762)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.090974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.096490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1261/2516] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:39.101799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.113524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0382804-217e-4d3d-aee2-595596906f1d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.118747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.119707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.119988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.120287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] PATH_INFO: `/v3/projects/ade50513f52b4de7b0a2909d1205a570/users/a5555f631e9a42d883cdcc38d54b1ce3/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.138405 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade50513f52b4de7b0a2909d1205a570, user_id=a5555f631e9a42d883cdcc38d54b1ce3, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.141071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.159553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b64e74629c4d6b9c5ef82247509762', 'request_id': 'req-8416b459-f232-4b91-b5d8-68f13f55b50a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:39.159822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1262/2517] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:39 2022] PUT /identity/v3/projects/ade50513f52b4de7b0a2909d1205a570/users/a5555f631e9a42d883cdcc38d54b1ce3/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:39.167026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.171559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b64e74629c4d6b9c5ef82247509762', 'request_id': 'req-8416b459-f232-4b91-b5d8-68f13f55b50a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:39.179534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b64e74629c4d6b9c5ef82247509762', 'request_id': 'req-8416b459-f232-4b91-b5d8-68f13f55b50a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:39.192687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] The token cache is being invalidate because project 03b64e74629c4d6b9c5ef82247509762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:47:39.193401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:39.194650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8416b459-f232-4b91-b5d8-68f13f55b50a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:47:39.196937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1256/2518] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:47:39 2022] DELETE /identity/v3/projects/03b64e74629c4d6b9c5ef82247509762 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:47:39.201936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901be8a-ee4c-4da4-baee-b898e5c485ee None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.202231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901be8a-ee4c-4da4-baee-b898e5c485ee None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.202510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901be8a-ee4c-4da4-baee-b898e5c485ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.226501 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d901be8a-ee4c-4da4-baee-b898e5c485ee None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:39.227784 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d901be8a-ee4c-4da4-baee-b898e5c485ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.254612 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973f150-bf53-402a-b657-2c8543bf6797 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.259644 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.260325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.260614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.260896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.261633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.263299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.280562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1263/2519] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:39.285422 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.290898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.310163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a103d1f-cc3d-4399-936f-ca05ac5c5b1c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.315034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d31b92-9d27-407a-b6b5-f117949b01ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.315947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d31b92-9d27-407a-b6b5-f117949b01ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.316236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d31b92-9d27-407a-b6b5-f117949b01ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.316510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d31b92-9d27-407a-b6b5-f117949b01ed admin admin] PATH_INFO: `/v3/projects/ade50513f52b4de7b0a2909d1205a570/users/a5555f631e9a42d883cdcc38d54b1ce3/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.318765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d31b92-9d27-407a-b6b5-f117949b01ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade50513f52b4de7b0a2909d1205a570, user_id=a5555f631e9a42d883cdcc38d54b1ce3, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.319559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1257/2520] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:39.323323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d31b92-9d27-407a-b6b5-f117949b01ed admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.327538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5553c15-c522-4b97-aa6c-79fa548aecb5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.327833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5553c15-c522-4b97-aa6c-79fa548aecb5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.328165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5553c15-c522-4b97-aa6c-79fa548aecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.342217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1264/2521] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:39 2022] PUT /identity/v3/projects/ade50513f52b4de7b0a2909d1205a570/users/a5555f631e9a42d883cdcc38d54b1ce3/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:47:39.348705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b5553c15-c522-4b97-aa6c-79fa548aecb5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:39.350382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b5553c15-c522-4b97-aa6c-79fa548aecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.354801 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.355095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.355378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.385493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] MFA Rules not processed for user `a5555f631e9a42d883cdcc38d54b1ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:39.386766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.433053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1258/2522] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:39.437707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5553c15-c522-4b97-aa6c-79fa548aecb5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.448397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5553c15-c522-4b97-aa6c-79fa548aecb5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.453811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.454810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.455103 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.455380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.456135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:47:39.456507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.458188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.461775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:39.474973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1259/2523] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:47:39.479511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:39.483408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.497163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ed4ba-ccce-4b17-aa30-41c24b16a6d9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.502501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1265/2524] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:47:39.506862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.507551 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.507834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.508108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.508812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.510780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.511696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.513118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.561374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1260/2525] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:39 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:47:39.562617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.578297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc292-44be-4df2-9eed-5b4b6e07d25c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.638917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ae6da-2b75-46a0-9ae8-d76d01fa7b5d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.644255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.644970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.645256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.645536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.646842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.648796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.655108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d995a543-9ea8-4713-890f-fd789a817509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.655784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995a543-9ea8-4713-890f-fd789a817509 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.656066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995a543-9ea8-4713-890f-fd789a817509 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.656345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d995a543-9ea8-4713-890f-fd789a817509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.658549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d995a543-9ea8-4713-890f-fd789a817509 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.675450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1261/2526] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:39 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:39.679773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.690867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e29fa-dc2b-4682-9f3e-4b90e718b0f7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.696792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.697764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.698252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.698744 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.700068 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.702328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.715508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1262/2527] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:39.719781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.730850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6a2e9-aa9e-4e8b-b907-adac08e2a5a2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.735768 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.736444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.736722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.736996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.737721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.739417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.750426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995a543-9ea8-4713-890f-fd789a817509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.751935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d995a543-9ea8-4713-890f-fd789a817509 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.753708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1263/2528] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:47:39.763491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1266/2529] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:47:39.764882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:39.780127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66804a0-9885-4213-a532-d13b04856ee9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:39.786252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:39.787008 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.787289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.787561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] PATH_INFO: `/v3/projects/f1832985606d4180930387dd35cb789d/users/7a775a256a624a97a55b08a0180dbadc/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.805182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1832985606d4180930387dd35cb789d, user_id=7a775a256a624a97a55b08a0180dbadc, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:39.807863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:39.824044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1264/2530] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:39 2022] PUT /identity/v3/projects/f1832985606d4180930387dd35cb789d/users/7a775a256a624a97a55b08a0180dbadc/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:39.835404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beacaa-3c57-4d44-acd6-a8c9fdfc1b00 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:39.835707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beacaa-3c57-4d44-acd6-a8c9fdfc1b00 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:39.836049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beacaa-3c57-4d44-acd6-a8c9fdfc1b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:39.873356 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-49beacaa-3c57-4d44-acd6-a8c9fdfc1b00 None None] MFA Rules not processed for user `7a775a256a624a97a55b08a0180dbadc`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:39.874949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-49beacaa-3c57-4d44-acd6-a8c9fdfc1b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:39.985172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:40.006731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1267/2531] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:40.008322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.088485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a13c1-03b2-4917-8bf1-bb348f6526af admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.093974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.094960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.095272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.095582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.096323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.098311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:40.152272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1265/2532] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:47:40 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:47:40.155555 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beacaa-3c57-4d44-acd6-a8c9fdfc1b00 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.167762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beacaa-3c57-4d44-acd6-a8c9fdfc1b00 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.278091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.279209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.279580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.279973 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.281237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.284224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:40.308229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1268/2533] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:47:40 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:47:40.312781 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.319006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:40.326517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d305fa-1b30-4d86-b176-249caf6000d5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.331668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.332347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.332640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.332918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.333673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.335381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:40.346846 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1266/2534] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:40.351896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.363871 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f2065-dcf3-43e2-a03e-d3ca6fb17f81 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.369271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.370236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.370552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.371186 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] PATH_INFO: `/v3/projects/839cc5c34b2e48138226d7bd75f0ac89/users/8fc67261978244aeae990718ebded99e/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.390734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839cc5c34b2e48138226d7bd75f0ac89, user_id=8fc67261978244aeae990718ebded99e, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.393415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:40.409901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1269/2535] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:40 2022] PUT /identity/v3/projects/839cc5c34b2e48138226d7bd75f0ac89/users/8fc67261978244aeae990718ebded99e/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:40.415326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.426658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799ceb7-13c1-4202-a55d-31ddead5cd06 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.495905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d08ebb-5668-4969-ab97-bac8a62f58f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.496581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d08ebb-5668-4969-ab97-bac8a62f58f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.496866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d08ebb-5668-4969-ab97-bac8a62f58f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.497146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d08ebb-5668-4969-ab97-bac8a62f58f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.497971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d08ebb-5668-4969-ab97-bac8a62f58f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.499650 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d08ebb-5668-4969-ab97-bac8a62f58f4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:40.510580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1267/2536] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:47:40.515054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.526592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df0d3-181d-483f-8ae4-7685fda55e24 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.532119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.532951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.533233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.533543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] PATH_INFO: `/v3/projects/839cc5c34b2e48138226d7bd75f0ac89/users/8fc67261978244aeae990718ebded99e/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.535567 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=839cc5c34b2e48138226d7bd75f0ac89, user_id=8fc67261978244aeae990718ebded99e, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.539209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:40.553663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1270/2537] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:47:40 2022] PUT /identity/v3/projects/839cc5c34b2e48138226d7bd75f0ac89/users/8fc67261978244aeae990718ebded99e/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:47:40.566496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d5ca-4d07-4489-b2a0-08ccdd764b07 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.566783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d5ca-4d07-4489-b2a0-08ccdd764b07 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.567066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574d5ca-4d07-4489-b2a0-08ccdd764b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.603992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7574d5ca-4d07-4489-b2a0-08ccdd764b07 None None] MFA Rules not processed for user `8fc67261978244aeae990718ebded99e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:47:40.605345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7574d5ca-4d07-4489-b2a0-08ccdd764b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:40.683660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1268/2538] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:47:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:47:40.700222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.772159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8f679-cc7c-4f62-8ee0-1de6fe6aff3f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.775873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.776524 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.777054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.777338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.795269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.796619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:40.807797 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1271/2539] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:40.894721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574d5ca-4d07-4489-b2a0-08ccdd764b07 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:40.906317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574d5ca-4d07-4489-b2a0-08ccdd764b07 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:40.910250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:40.911014 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:40.911337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:40.911614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:40.988038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:40.989504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:41.002641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1269/2540] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:44.601363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:47:44.617658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:47:44.651525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbcee5-8399-4216-b976-80f6cd7b9aa8 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:47:44.657978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372e315-ae87-4546-9617-9862b204ba92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:47:44.658657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372e315-ae87-4546-9617-9862b204ba92 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:47:44.658985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372e315-ae87-4546-9617-9862b204ba92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:47:44.659228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372e315-ae87-4546-9617-9862b204ba92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:47:44.747440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:44.783105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372e315-ae87-4546-9617-9862b204ba92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:47:44.786201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372e315-ae87-4546-9617-9862b204ba92 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:47:44.814545 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1272/2541] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:47:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:47:50.995219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:47:54.794528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:16.122508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:16.222055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6726b7d-2f2b-4b0d-bd5c-76eef9987fea service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:16.229668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:16.231467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:16.231634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:16.232047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] PATH_INFO: `/v3/users/27a55bbc7d8b4a489f8aa35af867298e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:16.234369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a55bbc7d8b4a489f8aa35af867298e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:16.238021 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:16.331698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a55bbc7d8b4a489f8aa35af867298e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:16.341729 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a55bbc7d8b4a489f8aa35af867298e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:16.352065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a55bbc7d8b4a489f8aa35af867298e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:16.352977 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a55bbc7d8b4a489f8aa35af867298e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:16.374418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a55bbc7d8b4a489f8aa35af867298e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:16.383381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1270/2542] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:16 2022] DELETE /identity/v3/users/27a55bbc7d8b4a489f8aa35af867298e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:16.389116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:16.389902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:16.389902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:16.414253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:16.415915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:16.500361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1273/2543] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:16.512070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:16.513965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:16.609874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbade21-94b9-4d5f-8151-19002921aa26 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:16.615901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:16.616957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:16.617529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:16.618109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:16.629563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:16.631084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:16.642385 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1271/2544] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:16.989130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:16.991168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:17.028830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e09fe-b9aa-4c6d-90f6-a72d6e50f23a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:17.038668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:17.040645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:17.041581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:17.042374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] PATH_INFO: `/v3/projects/dc35ed58a1614408b53949b6e2b10a8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:17.045004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc35ed58a1614408b53949b6e2b10a8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:17.050137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:17.146967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc35ed58a1614408b53949b6e2b10a8f', 'request_id': 'req-28623f0b-9641-425a-b773-1ba6b4b52880'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.159101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc35ed58a1614408b53949b6e2b10a8f', 'request_id': 'req-28623f0b-9641-425a-b773-1ba6b4b52880'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.173095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc35ed58a1614408b53949b6e2b10a8f', 'request_id': 'req-28623f0b-9641-425a-b773-1ba6b4b52880'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.195368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] The token cache is being invalidate because project dc35ed58a1614408b53949b6e2b10a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:17.196051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.196953 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.200607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1274/2545] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:16 2022] DELETE /identity/v3/projects/dc35ed58a1614408b53949b6e2b10a8f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:17.205614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:17.221964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df4b02-5797-4bd0-9d7b-fbbbfc94df2f service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:17.254866 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:17.352648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:17.354195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:17.354561 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:17.354733 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] PATH_INFO: `/v3/users/a5555f631e9a42d883cdcc38d54b1ce3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:17.356432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5555f631e9a42d883cdcc38d54b1ce3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:17.359829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:17.465277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5555f631e9a42d883cdcc38d54b1ce3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.474885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5555f631e9a42d883cdcc38d54b1ce3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.491369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5555f631e9a42d883cdcc38d54b1ce3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.492205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5555f631e9a42d883cdcc38d54b1ce3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.517046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3a14a307-fbb2-419e-8547-a5e183c07fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5555f631e9a42d883cdcc38d54b1ce3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:17.529966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1272/2546] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:17 2022] DELETE /identity/v3/users/a5555f631e9a42d883cdcc38d54b1ce3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:18.077202 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:18.079627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:18.105223 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28623f0b-9641-425a-b773-1ba6b4b52880 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:18.200124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:18.201018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.201536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.202033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] PATH_INFO: `/v3/projects/ade50513f52b4de7b0a2909d1205a570` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.204667 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ade50513f52b4de7b0a2909d1205a570)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:18.207517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:18.259664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:18.296387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ade50513f52b4de7b0a2909d1205a570', 'request_id': 'req-0ac9fb84-03bd-434f-aed6-b625687e8dc4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:18.312683 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ade50513f52b4de7b0a2909d1205a570', 'request_id': 'req-0ac9fb84-03bd-434f-aed6-b625687e8dc4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:18.325759 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ade50513f52b4de7b0a2909d1205a570', 'request_id': 'req-0ac9fb84-03bd-434f-aed6-b625687e8dc4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:18.347677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] The token cache is being invalidate because project ade50513f52b4de7b0a2909d1205a570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:18.348160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:18.349470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0ac9fb84-03bd-434f-aed6-b625687e8dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:18.352183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1275/2547] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:18 2022] DELETE /identity/v3/projects/ade50513f52b4de7b0a2909d1205a570 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:18.359978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d588f-0fac-47ab-8773-671ddc61e0f4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.360843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d588f-0fac-47ab-8773-671ddc61e0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.360843 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d588f-0fac-47ab-8773-671ddc61e0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.392603 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c26d588f-0fac-47ab-8773-671ddc61e0f4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:18.393858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c26d588f-0fac-47ab-8773-671ddc61e0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:18.481938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1273/2548] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:18.490575 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f010095-4324-4845-a5cd-818e9c9a0c2a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.490848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f010095-4324-4845-a5cd-818e9c9a0c2a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.491146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f010095-4324-4845-a5cd-818e9c9a0c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.526652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8f010095-4324-4845-a5cd-818e9c9a0c2a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:18.529167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8f010095-4324-4845-a5cd-818e9c9a0c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:18.552389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1276/2549] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:18.554110 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d588f-0fac-47ab-8773-671ddc61e0f4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:18.566539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d588f-0fac-47ab-8773-671ddc61e0f4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:18.574584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:18.575765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.576265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.576764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.578010 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:18.578342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:18.580471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:18.595005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1274/2550] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:18.600301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f010095-4324-4845-a5cd-818e9c9a0c2a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:18.610235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f010095-4324-4845-a5cd-818e9c9a0c2a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:18.615988 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:18.616657 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.616943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.617269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.618013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:18.626001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:18.673909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1277/2551] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:18 2022] POST /identity/v3/projects => generated 393 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:18.679274 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:18.697303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7288c-ea7d-4fc3-a6b7-c82e37519317 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:18.745346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:18.795828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:18.796528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.796812 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.797093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.797830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:18.799805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:18.836011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1275/2552] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:18 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:18.841325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:18.856712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487fea9-c9cb-4789-aa3e-d95c9758c7f5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:18.867201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:18.871874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.871874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.871874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.871874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:18.874420 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:18.892290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1278/2553] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:18.899505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:18.920007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc76b-9ec7-4dec-8583-c6beafc8859f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:18.928231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5475b-f298-4bbe-8427-267f58c1d833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:18.929332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5475b-f298-4bbe-8427-267f58c1d833 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.929804 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5475b-f298-4bbe-8427-267f58c1d833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.930318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5475b-f298-4bbe-8427-267f58c1d833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:18.931633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5475b-f298-4bbe-8427-267f58c1d833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:18.934243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5475b-f298-4bbe-8427-267f58c1d833 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:18.953026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1276/2554] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:18.954765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:18.973875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07199a0b-6060-41c8-89e2-12eb91914916 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:18.979341 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:18.980491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:18.980959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:18.981392 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] PATH_INFO: `/v3/projects/800f3c1bab894bcba0a9c4c940939084/users/85355dfda6eb44e0a5c09bb3bede50ce/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.009529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=800f3c1bab894bcba0a9c4c940939084, user_id=85355dfda6eb44e0a5c09bb3bede50ce, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.013787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.037586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1279/2555] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:18 2022] PUT /identity/v3/projects/800f3c1bab894bcba0a9c4c940939084/users/85355dfda6eb44e0a5c09bb3bede50ce/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:19.050939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5a12a-59c0-40a4-84c2-8ab129e91619 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.051242 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5a12a-59c0-40a4-84c2-8ab129e91619 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.051528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5a12a-59c0-40a4-84c2-8ab129e91619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.088253 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-38f5a12a-59c0-40a4-84c2-8ab129e91619 None None] MFA Rules not processed for user `85355dfda6eb44e0a5c09bb3bede50ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:19.090277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-38f5a12a-59c0-40a4-84c2-8ab129e91619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:19.195931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1277/2556] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:19.201243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.307007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e847198-5e7b-423a-8057-df801159a07f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.316816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.317402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.317915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.318401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.319380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.322184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.381068 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1280/2557] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:19 2022] POST /identity/v3/projects => generated 389 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:19.383357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5a12a-59c0-40a4-84c2-8ab129e91619 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.398086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5a12a-59c0-40a4-84c2-8ab129e91619 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.510348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.511611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.511997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.512426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.513386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.515938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.553121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1278/2558] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:19 2022] POST /identity/v3/users => generated 301 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:19.559218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.571212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccad2b-f2f2-4d5f-95ef-15c467e47472 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.581396 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.582652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.582935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.583211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.583959 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.585572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.599576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1281/2559] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:19.605417 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.617116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1943a-f4ad-4c31-92ba-75d70b0bfdff admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.624004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.625044 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.625447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.625817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] PATH_INFO: `/v3/projects/f9aca00a4a474126a9d37bbc5ac69a5c/users/a15636e2422a448d88b8497ade236f4b/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.648036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9aca00a4a474126a9d37bbc5ac69a5c, user_id=a15636e2422a448d88b8497ade236f4b, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.652158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.672476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1279/2560] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:19 2022] PUT /identity/v3/projects/f9aca00a4a474126a9d37bbc5ac69a5c/users/a15636e2422a448d88b8497ade236f4b/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:19.677366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.688265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745eaeba-6f5e-4ea2-bf05-352620faa591 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.717782 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:19.779691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80435322-df6b-4762-ac4d-ae7bcc21e096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.780663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80435322-df6b-4762-ac4d-ae7bcc21e096 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.781224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80435322-df6b-4762-ac4d-ae7bcc21e096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.781604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80435322-df6b-4762-ac4d-ae7bcc21e096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.782448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80435322-df6b-4762-ac4d-ae7bcc21e096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.784867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80435322-df6b-4762-ac4d-ae7bcc21e096 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.797887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1282/2561] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:19.802605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:19.811738 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:19.822165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e951b6-c345-4728-9153-6c128d7bed7e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:19.830623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:19.832723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.833223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.833711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] PATH_INFO: `/v3/projects/f9aca00a4a474126a9d37bbc5ac69a5c/users/a15636e2422a448d88b8497ade236f4b/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.837218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9aca00a4a474126a9d37bbc5ac69a5c, user_id=a15636e2422a448d88b8497ade236f4b, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:19.840760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:19.860236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1280/2562] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:19 2022] PUT /identity/v3/projects/f9aca00a4a474126a9d37bbc5ac69a5c/users/a15636e2422a448d88b8497ade236f4b/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:19.877751 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16f94-110d-47da-8332-e1093644a038 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:19.878299 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16f94-110d-47da-8332-e1093644a038 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:19.878866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16f94-110d-47da-8332-e1093644a038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:19.927592 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-59d16f94-110d-47da-8332-e1093644a038 None None] MFA Rules not processed for user `a15636e2422a448d88b8497ade236f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:19.928866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-59d16f94-110d-47da-8332-e1093644a038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:20.025986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1283/2563] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:20.045286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:20.048030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:20.137314 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0065d-855e-48c0-88e1-e918bcc1ea95 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:20.141974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:20.142582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:20.142844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:20.143121 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:20.157410 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:20.158972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:20.172869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1281/2564] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:20.255162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d16f94-110d-47da-8332-e1093644a038 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:20.273796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d16f94-110d-47da-8332-e1093644a038 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:20.280207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:20.281350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:20.281659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:20.282025 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:20.395316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:20.395316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:20.406663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1284/2565] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:20.724611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:20.727921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:20.811435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090827-4c10-480d-ada8-eb4e24a66947 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:20.817508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c90d2ca-95fd-47f1-a05d-0ed2ddc3693c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:20.818899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90d2ca-95fd-47f1-a05d-0ed2ddc3693c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:20.819574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90d2ca-95fd-47f1-a05d-0ed2ddc3693c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:20.820183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c90d2ca-95fd-47f1-a05d-0ed2ddc3693c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:20.825217 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7c90d2ca-95fd-47f1-a05d-0ed2ddc3693c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:20.932421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90d2ca-95fd-47f1-a05d-0ed2ddc3693c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:20.933919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c90d2ca-95fd-47f1-a05d-0ed2ddc3693c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:20.945211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1282/2566] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:24.447510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:24.482245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7c87-6767-4c29-bbc9-11c62c386be2 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:24.490418 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:24.492078 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:24.492580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:24.492897 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] PATH_INFO: `/v3/users/85355dfda6eb44e0a5c09bb3bede50ce` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:24.495424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=85355dfda6eb44e0a5c09bb3bede50ce)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:24.498106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:24.557708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:24.568349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85355dfda6eb44e0a5c09bb3bede50ce'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:24.581038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85355dfda6eb44e0a5c09bb3bede50ce'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:24.591803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85355dfda6eb44e0a5c09bb3bede50ce'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:24.592674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85355dfda6eb44e0a5c09bb3bede50ce'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:24.618743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85355dfda6eb44e0a5c09bb3bede50ce'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:24.640807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1285/2567] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:24 2022] DELETE /identity/v3/users/85355dfda6eb44e0a5c09bb3bede50ce => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:24.645861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:24.645861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:24.645861 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:24.689797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:24.692515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:24.772301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:24.810888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1283/2568] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:24.829394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:24.833471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:24.954716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08deff-8406-4b70-b345-5a1ff18ca62c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:24.960099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:24.961196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:24.961651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:24.962107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:24.977398 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:24.979437 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:24.994943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1286/2569] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:25.512045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.513721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:25.549788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea94e-508e-41e6-b43a-240825e8affa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.558694 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.560764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.560764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.561118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] PATH_INFO: `/v3/users/9b2a8aa8f4724e48a41afce944657ba5` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.563685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2a8aa8f4724e48a41afce944657ba5)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.573257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.676909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2a8aa8f4724e48a41afce944657ba5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.689421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2a8aa8f4724e48a41afce944657ba5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.700712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2a8aa8f4724e48a41afce944657ba5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.701447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2a8aa8f4724e48a41afce944657ba5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.713425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.719893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:25.721696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2a8aa8f4724e48a41afce944657ba5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.743538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1284/2570] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:25 2022] DELETE /identity/v3/users/9b2a8aa8f4724e48a41afce944657ba5 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:25.748660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531ab7f-d89f-48b5-b7d7-14fd5f2932d7 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:25.838901 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:25.839793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:25.840094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:25.840359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] PATH_INFO: `/v3/projects/800f3c1bab894bcba0a9c4c940939084` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:25.841707 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=800f3c1bab894bcba0a9c4c940939084)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:25.843702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:25.904245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800f3c1bab894bcba0a9c4c940939084', 'request_id': 'req-d15bbe64-aa92-4236-b2ff-aba117e755ff'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.917639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800f3c1bab894bcba0a9c4c940939084', 'request_id': 'req-d15bbe64-aa92-4236-b2ff-aba117e755ff'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.925413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:25.931324 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800f3c1bab894bcba0a9c4c940939084', 'request_id': 'req-d15bbe64-aa92-4236-b2ff-aba117e755ff'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.954316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] The token cache is being invalidate because project 800f3c1bab894bcba0a9c4c940939084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:25.954787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.955928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:25.959684 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1287/2571] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:25 2022] DELETE /identity/v3/projects/800f3c1bab894bcba0a9c4c940939084 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:25.965742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:25.968946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:25.992710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be11d2-28e2-4026-a9fd-b1529cbe1154 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.043069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.045778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:26.103379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.104421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.104799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.105206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] PATH_INFO: `/v3/users/a15636e2422a448d88b8497ade236f4b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.106938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15636e2422a448d88b8497ade236f4b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.108989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.170287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bbe64-aa92-4236-b2ff-aba117e755ff admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.179723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.181021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.181504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.182044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] PATH_INFO: `/v3/projects/a0f2b3c7fb15462d8a230d488534fe18` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.184306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f2b3c7fb15462d8a230d488534fe18)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.186651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.194604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15636e2422a448d88b8497ade236f4b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.214362 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15636e2422a448d88b8497ade236f4b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.216919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:26.231980 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15636e2422a448d88b8497ade236f4b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.233415 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15636e2422a448d88b8497ade236f4b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.258158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-47376be7-0533-461a-b0aa-411e21bd1e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15636e2422a448d88b8497ade236f4b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.272204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1285/2572] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:25 2022] DELETE /identity/v3/users/a15636e2422a448d88b8497ade236f4b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:26.285026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f2b3c7fb15462d8a230d488534fe18', 'request_id': 'req-07c3fe5a-ffff-499a-9b43-309de170ad5a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.297662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f2b3c7fb15462d8a230d488534fe18', 'request_id': 'req-07c3fe5a-ffff-499a-9b43-309de170ad5a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.310252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f2b3c7fb15462d8a230d488534fe18', 'request_id': 'req-07c3fe5a-ffff-499a-9b43-309de170ad5a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.337712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] The token cache is being invalidate because project a0f2b3c7fb15462d8a230d488534fe18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:26.338210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.339329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07c3fe5a-ffff-499a-9b43-309de170ad5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:26.342562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1288/2573] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:26 2022] DELETE /identity/v3/projects/a0f2b3c7fb15462d8a230d488534fe18 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:26.351030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0a3e8-cb8f-4be2-9253-1cd9c28cce1e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.351694 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0a3e8-cb8f-4be2-9253-1cd9c28cce1e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.352158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0a3e8-cb8f-4be2-9253-1cd9c28cce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.385985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6bb0a3e8-cb8f-4be2-9253-1cd9c28cce1e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:26.388471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6bb0a3e8-cb8f-4be2-9253-1cd9c28cce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:26.485183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1286/2574] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:26.499084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b2ae-f05c-4cba-a6ea-4273547063c2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.499512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b2ae-f05c-4cba-a6ea-4273547063c2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.500689 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73b2ae-f05c-4cba-a6ea-4273547063c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.525551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8b73b2ae-f05c-4cba-a6ea-4273547063c2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:26.527107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8b73b2ae-f05c-4cba-a6ea-4273547063c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:26.544476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1289/2575] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:26.546338 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0a3e8-cb8f-4be2-9253-1cd9c28cce1e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.563432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0a3e8-cb8f-4be2-9253-1cd9c28cce1e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.569401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.570135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.570419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.570707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.571454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:26.571774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.573426 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.588821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1287/2576] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:26.595336 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b2ae-f05c-4cba-a6ea-4273547063c2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.613510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73b2ae-f05c-4cba-a6ea-4273547063c2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.626457 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.629141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.629950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.630650 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.632257 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.636643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.694196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1290/2577] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:26 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:26.699955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.717567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561be8f-f4f5-4996-a68e-6ea86e7ec761 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.752702 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.756548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:26.851200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.851895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.852182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.852467 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.853153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.855343 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.889558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1288/2578] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:26 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:26.891957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.900764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:26.906548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213ef7e-1508-44c2-ad92-0bf98053aceb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.919613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.921037 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.921622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.922223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.923291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0e4d3-1ec0-4326-882f-83fbd0d2bb2c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.923801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.928731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.932895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.934845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.934845 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.935217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] PATH_INFO: `/v3/projects/f9aca00a4a474126a9d37bbc5ac69a5c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.936898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9aca00a4a474126a9d37bbc5ac69a5c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.939868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:26.951342 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1289/2579] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:26.952122 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:26.975166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2060de4-54b7-4beb-a4f2-da112ff3de04 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:26.982361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:26.983065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:26.983348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:26.983628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:26.984457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:26.986370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.003052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1290/2580] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:27.007384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.018509 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae71cb-876c-4467-a506-1f68d8149e10 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.036725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9aca00a4a474126a9d37bbc5ac69a5c', 'request_id': 'req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.057702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:27.060522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9aca00a4a474126a9d37bbc5ac69a5c', 'request_id': 'req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.076869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9aca00a4a474126a9d37bbc5ac69a5c', 'request_id': 'req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.095780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] The token cache is being invalidate because project f9aca00a4a474126a9d37bbc5ac69a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:27.096387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.097825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a3d53ab2-bce5-471d-9fdf-96fecc778ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:27.099635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1291/2581] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:26 2022] DELETE /identity/v3/projects/f9aca00a4a474126a9d37bbc5ac69a5c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:27.114734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f7988-cfd8-442f-ac7f-a9117a442d90 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.115111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f7988-cfd8-442f-ac7f-a9117a442d90 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.115590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f7988-cfd8-442f-ac7f-a9117a442d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.133699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca620ff-2cee-40dc-a9ea-25d5b7869616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.134900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca620ff-2cee-40dc-a9ea-25d5b7869616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.136034 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca620ff-2cee-40dc-a9ea-25d5b7869616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.136424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca620ff-2cee-40dc-a9ea-25d5b7869616 admin admin] PATH_INFO: `/v3/projects/180bdc5014064079b8f5ac753ac72dd9/users/e967f760bdbd43488ae8d623e20b4405/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.138587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-af0f7988-cfd8-442f-ac7f-a9117a442d90 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:27.139753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-af0f7988-cfd8-442f-ac7f-a9117a442d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:27.155979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca620ff-2cee-40dc-a9ea-25d5b7869616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=180bdc5014064079b8f5ac753ac72dd9, user_id=e967f760bdbd43488ae8d623e20b4405, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.158907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca620ff-2cee-40dc-a9ea-25d5b7869616 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.162031 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1292/2582] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:27.175206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27387ed8-3631-4db8-904d-718462496248 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.175905 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27387ed8-3631-4db8-904d-718462496248 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.177172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27387ed8-3631-4db8-904d-718462496248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.196321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1291/2583] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:27 2022] PUT /identity/v3/projects/180bdc5014064079b8f5ac753ac72dd9/users/e967f760bdbd43488ae8d623e20b4405/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:27.206198 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79bc4e-4093-41a5-b5de-937f9ef01135 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.207271 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-27387ed8-3631-4db8-904d-718462496248 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:27.207464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79bc4e-4093-41a5-b5de-937f9ef01135 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.207873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79bc4e-4093-41a5-b5de-937f9ef01135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.209252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-27387ed8-3631-4db8-904d-718462496248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:27.255380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8f79bc4e-4093-41a5-b5de-937f9ef01135 None None] MFA Rules not processed for user `e967f760bdbd43488ae8d623e20b4405`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:27.257329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8f79bc4e-4093-41a5-b5de-937f9ef01135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:27.317376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1293/2584] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:27.322023 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27387ed8-3631-4db8-904d-718462496248 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.335540 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27387ed8-3631-4db8-904d-718462496248 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.344055 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.345222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.345745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.346288 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.348309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:27.348875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.351836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.366907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1292/2585] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:27.370907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1294/2586] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:27.375715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79bc4e-4093-41a5-b5de-937f9ef01135 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.385294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.388001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:27.393085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79bc4e-4093-41a5-b5de-937f9ef01135 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.401897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.402986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.403446 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.403907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.405498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.408857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.447374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1293/2587] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:27.452204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.484842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:27.512151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa976eb-8e4b-4743-ae07-b5e3499416c0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.551197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144735de-4ed5-4735-a716-36c93bf4568b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.556699 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.557425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.557710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.558027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.558732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.564335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.592456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1294/2588] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:27.594299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.606132 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbba4b-4fb3-4e23-b78a-c04b8aeb31df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.607170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbba4b-4fb3-4e23-b78a-c04b8aeb31df service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.607621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbba4b-4fb3-4e23-b78a-c04b8aeb31df service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.608071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbba4b-4fb3-4e23-b78a-c04b8aeb31df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.611313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc180f-b1ad-47d1-b89c-2225935a287a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.619435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.620527 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.621011 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.621488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.622659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.625088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.630638 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:27.653694 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1295/2589] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:27.654889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.665463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052df921-8c9b-4f67-86e1-8135c9739189 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.670711 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.671584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.671978 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.672356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.673286 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.675404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.690281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1296/2590] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:27.692086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:27.694714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbba4b-4fb3-4e23-b78a-c04b8aeb31df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.696599 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbba4b-4fb3-4e23-b78a-c04b8aeb31df service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.703631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac36c-239e-44ba-8020-6d48a2d37b1f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:27.710117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:27.711181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.711345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1295/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:27.713985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.714547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] PATH_INFO: `/v3/projects/03330ec7d53e46efac5a93a41e1eb8f6/users/cd2cf5166fe74cdf8399e7eb2873f518/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.752028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] RBAC: Authorizing `identity:create_grant(project_id=03330ec7d53e46efac5a93a41e1eb8f6, user_id=cd2cf5166fe74cdf8399e7eb2873f518, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:27.756469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:27.780971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1297/2592] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:27 2022] PUT /identity/v3/projects/03330ec7d53e46efac5a93a41e1eb8f6/users/cd2cf5166fe74cdf8399e7eb2873f518/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:27.803007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d08bd-55ad-42af-8921-441520e131da None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:27.803007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d08bd-55ad-42af-8921-441520e131da None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:27.803007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d08bd-55ad-42af-8921-441520e131da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:27.839248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a84d08bd-55ad-42af-8921-441520e131da None None] MFA Rules not processed for user `cd2cf5166fe74cdf8399e7eb2873f518`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:27.840477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a84d08bd-55ad-42af-8921-441520e131da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:27.958739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1296/2593] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:27.963668 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.020389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d08bd-55ad-42af-8921-441520e131da None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.022085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a84d08bd-55ad-42af-8921-441520e131da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:28.061119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feef7e5-0a27-47b4-a9e8-24bcab2eaeae admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.065406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:28.067548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.068356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.068696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.068999 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.069700 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.071686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.103033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1298/2594] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:27 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:28.107596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.127656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d08bd-55ad-42af-8921-441520e131da None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.218158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceced395-a8ac-428e-8126-248cfc3fc0c0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.225657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.226783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.227256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.227732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.228888 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.230107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.231070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.231266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.231440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.231826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] PATH_INFO: `/v3/users/e967f760bdbd43488ae8d623e20b4405` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.233204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e967f760bdbd43488ae8d623e20b4405)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.235196 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.266336 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1299/2595] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:28 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:28.267974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.287493 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f2ee4-674c-434d-bf9d-69eb7a4b334f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.296492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.297621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.298151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.298623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.299983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.306808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.323437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1300/2596] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:28.328504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.337668 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e967f760bdbd43488ae8d623e20b4405'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.344298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94c9e4-1c0e-4226-ad0d-8924aac807bc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.349149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e967f760bdbd43488ae8d623e20b4405'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.353003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.354672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.355267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.355630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] PATH_INFO: `/v3/projects/11c9fdfdc9334939ac898e9b5ef6d35b/users/a9bf44dc8d574e4bb92b82ad527e3da9/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.364752 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e967f760bdbd43488ae8d623e20b4405'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.366173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e967f760bdbd43488ae8d623e20b4405'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.380612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c9fdfdc9334939ac898e9b5ef6d35b, user_id=a9bf44dc8d574e4bb92b82ad527e3da9, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.383827 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.386245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c66218f-ef28-4b9a-bde5-5f40fa60dd71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e967f760bdbd43488ae8d623e20b4405'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:28.399672 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1297/2597] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:28 2022] DELETE /identity/v3/users/e967f760bdbd43488ae8d623e20b4405 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:28.405072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1301/2598] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:28 2022] PUT /identity/v3/projects/11c9fdfdc9334939ac898e9b5ef6d35b/users/a9bf44dc8d574e4bb92b82ad527e3da9/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:28.408028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.409122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.410286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.411775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.414872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:28.423658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:28.438537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:28.440497 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:28.512439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:28.540923 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1298/2599] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:28.545277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37abd1-7076-4312-a7ea-70905f957271 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.554488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.555707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.556184 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.556528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.557475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.559726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.561154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.563610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:28.584028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1302/2600] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:28.586028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.604519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7dc86-38e0-48c4-bc47-9f1547b2a081 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.612304 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb57b4-a515-403f-b18d-c93e73d86b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.613669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb57b4-a515-403f-b18d-c93e73d86b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.614133 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb57b4-a515-403f-b18d-c93e73d86b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.614578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb57b4-a515-403f-b18d-c93e73d86b00 admin admin] PATH_INFO: `/v3/projects/11c9fdfdc9334939ac898e9b5ef6d35b/users/a9bf44dc8d574e4bb92b82ad527e3da9/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.617018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb57b4-a515-403f-b18d-c93e73d86b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c9fdfdc9334939ac898e9b5ef6d35b, user_id=a9bf44dc8d574e4bb92b82ad527e3da9, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.621722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb57b4-a515-403f-b18d-c93e73d86b00 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.645238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1303/2601] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:28 2022] PUT /identity/v3/projects/11c9fdfdc9334939ac898e9b5ef6d35b/users/a9bf44dc8d574e4bb92b82ad527e3da9/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:28.664889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.665381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.666409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.689351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60675d8-204a-45be-aae4-49b9b2072c09 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.712949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] MFA Rules not processed for user `a9bf44dc8d574e4bb92b82ad527e3da9`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:28.714320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:28.770944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.771837 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.772124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.772439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.815681 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1304/2602] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:28.828301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:28.831097 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:28.863982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.866087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.881450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1299/2603] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:28.931326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66daca0d-0a9d-4440-a702-cc3baf117469 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:28.938321 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69119b21-6ad0-4c44-80fa-ea04090898a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:28.939072 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69119b21-6ad0-4c44-80fa-ea04090898a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:28.939448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69119b21-6ad0-4c44-80fa-ea04090898a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:28.939911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69119b21-6ad0-4c44-80fa-ea04090898a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:28.955801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69119b21-6ad0-4c44-80fa-ea04090898a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:28.957869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69119b21-6ad0-4c44-80fa-ea04090898a4 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:28.981822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1305/2604] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:29.280102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.280102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.303453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fb681-62d9-45ea-979f-c3ca3ae0e119 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:29.310509 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:29.311933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.312339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.312690 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] PATH_INFO: `/v3/projects/180bdc5014064079b8f5ac753ac72dd9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.314165 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=180bdc5014064079b8f5ac753ac72dd9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:29.316278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:29.385044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180bdc5014064079b8f5ac753ac72dd9', 'request_id': 'req-f00f5887-aa45-4f4e-bc86-eca49e670401'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.394351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:29.400632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180bdc5014064079b8f5ac753ac72dd9', 'request_id': 'req-f00f5887-aa45-4f4e-bc86-eca49e670401'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.416850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180bdc5014064079b8f5ac753ac72dd9', 'request_id': 'req-f00f5887-aa45-4f4e-bc86-eca49e670401'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.434160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] The token cache is being invalidate because project 180bdc5014064079b8f5ac753ac72dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:29.435348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.436842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.438698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1300/2605] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:29 2022] DELETE /identity/v3/projects/180bdc5014064079b8f5ac753ac72dd9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:29.448339 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475c71-b127-4331-b6e1-9b88a6b53c30 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.448842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475c71-b127-4331-b6e1-9b88a6b53c30 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.449335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c475c71-b127-4331-b6e1-9b88a6b53c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.471854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7c475c71-b127-4331-b6e1-9b88a6b53c30 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:29.473267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7c475c71-b127-4331-b6e1-9b88a6b53c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.483377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:29.548843 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.551551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.576127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1306/2606] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:29.585788 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab505b5-b8aa-4efe-936f-3939d8d736dc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.586261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab505b5-b8aa-4efe-936f-3939d8d736dc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.586697 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab505b5-b8aa-4efe-936f-3939d8d736dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.609107 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8ab505b5-b8aa-4efe-936f-3939d8d736dc None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:29.610496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8ab505b5-b8aa-4efe-936f-3939d8d736dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.619615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1307/2607] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:29.623762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab505b5-b8aa-4efe-936f-3939d8d736dc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.642918 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab505b5-b8aa-4efe-936f-3939d8d736dc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:29.651937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f5887-aa45-4f4e-bc86-eca49e670401 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:29.652161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:29.653376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.654018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.654634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.656147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:29.656717 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:29.658384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:29.659382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.659697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.659873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:29.660094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] PATH_INFO: `/v3/users/cd2cf5166fe74cdf8399e7eb2873f518` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.661714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2cf5166fe74cdf8399e7eb2873f518)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:29.664525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:29.680386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1308/2608] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:29.684976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.701176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6a61f-49e4-4bb9-a79d-a24fb3c285f0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:29.711411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:29.712723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.713272 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.713821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.715194 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:29.718982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:29.749829 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2cf5166fe74cdf8399e7eb2873f518'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.761548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2cf5166fe74cdf8399e7eb2873f518'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.766528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1309/2609] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:29 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:29.774839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.775073 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2cf5166fe74cdf8399e7eb2873f518'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.776197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2cf5166fe74cdf8399e7eb2873f518'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.799578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f23e9d35-1b7f-4883-8946-fac40c84a924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2cf5166fe74cdf8399e7eb2873f518'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:29.812174 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1301/2610] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:29 2022] DELETE /identity/v3/users/cd2cf5166fe74cdf8399e7eb2873f518 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:29.820885 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:29.821345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:29.821748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:29.853577 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:29.854879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.893178 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bd499-21b6-4653-8ba1-67d42a091534 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:29.961474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1302/2611] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:29.978840 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:29.980918 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:29.990178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:30.012856 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.013798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.014297 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.014608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.015718 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.019554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.060025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1310/2612] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:29 2022] POST /identity/v3/users => generated 290 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:30.066065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.068722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.077103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed23a8-6eae-4423-b3aa-3d9a47230b54 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.084513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.085804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.086609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.087044 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.093710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630bfa8-3119-414e-8105-13982d20441c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.103130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.103576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.104686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.105151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.105702 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.106277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.107142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.108972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.115526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:30.125866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1303/2613] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:30.137695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1311/2614] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:30.140563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.167255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fc1e6-7b64-42dc-90e3-96028065d26d service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.178631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008ad8-b09f-409e-96ab-dfea9ac96227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.179821 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008ad8-b09f-409e-96ab-dfea9ac96227 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.180306 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008ad8-b09f-409e-96ab-dfea9ac96227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.181415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008ad8-b09f-409e-96ab-dfea9ac96227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.183236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee008ad8-b09f-409e-96ab-dfea9ac96227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.186597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee008ad8-b09f-409e-96ab-dfea9ac96227 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.218976 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1304/2615] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:30.221606 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.239260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4598-0393-458c-ba50-5374fb6dc9ce admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.248464 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.250100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.250909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.251575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] PATH_INFO: `/v3/projects/6d7dd56eefed455f807a41990f358efe/users/f8418981b69d491da8c5de490b71f898/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.280754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7dd56eefed455f807a41990f358efe, user_id=f8418981b69d491da8c5de490b71f898, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.286532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.313046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1312/2616] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:30 2022] PUT /identity/v3/projects/6d7dd56eefed455f807a41990f358efe/users/f8418981b69d491da8c5de490b71f898/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:30.332282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772b2d-a7b5-4151-96f9-7059eddd2949 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.332664 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772b2d-a7b5-4151-96f9-7059eddd2949 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.332869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772b2d-a7b5-4151-96f9-7059eddd2949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.386999 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-be772b2d-a7b5-4151-96f9-7059eddd2949 None None] MFA Rules not processed for user `f8418981b69d491da8c5de490b71f898`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:30.389625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-be772b2d-a7b5-4151-96f9-7059eddd2949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.408553 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.411752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.508917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1305/2617] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:30.514817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e4636-de82-4cbc-a914-a4f3cb125798 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.515092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772b2d-a7b5-4151-96f9-7059eddd2949 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.522619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.524070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.524469 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.524849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] PATH_INFO: `/v3/users/7a775a256a624a97a55b08a0180dbadc` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.526582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a775a256a624a97a55b08a0180dbadc)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.528647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.531828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772b2d-a7b5-4151-96f9-7059eddd2949 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.538842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.540081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.540703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.541415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.542419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.545419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.592949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1306/2618] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:30 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:30.594536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.600134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.639703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a775a256a624a97a55b08a0180dbadc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.656660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a775a256a624a97a55b08a0180dbadc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.668106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a775a256a624a97a55b08a0180dbadc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.669074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a775a256a624a97a55b08a0180dbadc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.688504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a775a256a624a97a55b08a0180dbadc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.702252 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1313/2619] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:30 2022] DELETE /identity/v3/users/7a775a256a624a97a55b08a0180dbadc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:30.705535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.706059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.723075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e5512-85bc-461c-99fb-5d8eea5ae178 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.731163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:30.787899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:30.804232 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d725863-b0c9-4aee-bad7-18ec3fabe457 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:30.812800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3543a8d-aa5d-4c86-8fd8-882d90dc18bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.813507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543a8d-aa5d-4c86-8fd8-882d90dc18bc admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.813797 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543a8d-aa5d-4c86-8fd8-882d90dc18bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.814114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3543a8d-aa5d-4c86-8fd8-882d90dc18bc admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.814835 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3543a8d-aa5d-4c86-8fd8-882d90dc18bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.817440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3543a8d-aa5d-4c86-8fd8-882d90dc18bc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.829808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:30.831329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.831620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.831899 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] PATH_INFO: `/v3/projects/03330ec7d53e46efac5a93a41e1eb8f6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.833483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=03330ec7d53e46efac5a93a41e1eb8f6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:30.835429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:30.848779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1314/2620] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:30 2022] POST /identity/v3/users => generated 297 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:30.848779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:30.848779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:30.848779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:30.876247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:30.879202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.894624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03330ec7d53e46efac5a93a41e1eb8f6', 'request_id': 'req-b3128c23-090b-4a24-8b1b-e1770a565b0f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.908439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03330ec7d53e46efac5a93a41e1eb8f6', 'request_id': 'req-b3128c23-090b-4a24-8b1b-e1770a565b0f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.920170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03330ec7d53e46efac5a93a41e1eb8f6', 'request_id': 'req-b3128c23-090b-4a24-8b1b-e1770a565b0f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.938938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] The token cache is being invalidate because project 03330ec7d53e46efac5a93a41e1eb8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:30.939209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.939917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:30.942123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1307/2621] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:30 2022] DELETE /identity/v3/projects/03330ec7d53e46efac5a93a41e1eb8f6 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:30.942945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.944383 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:30.989376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1315/2622] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:30.990992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:30.993088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.013590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e01a33-ba0d-4083-b45c-8c6c4f1323ff None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.018908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.019816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.020102 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.020389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] PATH_INFO: `/v3/users/a9bf44dc8d574e4bb92b82ad527e3da9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.021672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9bf44dc8d574e4bb92b82ad527e3da9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.023672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.052906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3128c23-090b-4a24-8b1b-e1770a565b0f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.060293 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.061331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.061740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.062159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.063318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.065617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.087008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1308/2623] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:31.087008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.088117 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.090646 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9bf44dc8d574e4bb92b82ad527e3da9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.099117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9bf44dc8d574e4bb92b82ad527e3da9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.109785 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9bf44dc8d574e4bb92b82ad527e3da9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.110775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9bf44dc8d574e4bb92b82ad527e3da9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.129605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9bf44dc8d574e4bb92b82ad527e3da9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.140482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1316/2624] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:30 2022] DELETE /identity/v3/users/a9bf44dc8d574e4bb92b82ad527e3da9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:31.142082 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.144558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.155698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:31.192832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b42990-ba04-4e9f-8c61-50ce68fae19d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.264989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:31.277701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97286a-8aac-4885-bee8-fb9965e2a778 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.290432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.292909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.293947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.294162 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.295138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.295365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.295572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.295735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] PATH_INFO: `/v3/projects/cb370de8c0c240fe9202df5230c65e49/users/853b4db3b1cb4d6db1af5803ac376801/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.298308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.324495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb370de8c0c240fe9202df5230c65e49, user_id=853b4db3b1cb4d6db1af5803ac376801, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.324778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.327713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.331443 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.346181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1309/2625] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:31.357256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1317/2626] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:31 2022] PUT /identity/v3/projects/cb370de8c0c240fe9202df5230c65e49/users/853b4db3b1cb4d6db1af5803ac376801/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:31.364088 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.382127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb111d-257e-445c-b71f-85eb30dd75f2 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.514151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9916b-3fa7-4d6b-821c-4a12d3bdc3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.515297 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9916b-3fa7-4d6b-821c-4a12d3bdc3ff admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.515796 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9916b-3fa7-4d6b-821c-4a12d3bdc3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.516287 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9916b-3fa7-4d6b-821c-4a12d3bdc3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.517887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9916b-3fa7-4d6b-821c-4a12d3bdc3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.521731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9916b-3fa7-4d6b-821c-4a12d3bdc3ff admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.548958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1310/2627] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:31.550981 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.564207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30834ab-4c96-45fd-bb0c-6d8425d32b5c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.574480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.575669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.576013 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.576364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] PATH_INFO: `/v3/projects/cb370de8c0c240fe9202df5230c65e49/users/853b4db3b1cb4d6db1af5803ac376801/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.578646 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb370de8c0c240fe9202df5230c65e49, user_id=853b4db3b1cb4d6db1af5803ac376801, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.582038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.599332 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1318/2628] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:31 2022] PUT /identity/v3/projects/cb370de8c0c240fe9202df5230c65e49/users/853b4db3b1cb4d6db1af5803ac376801/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:31.618842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.619421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.620844 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.672686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.676289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.704127 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] MFA Rules not processed for user `853b4db3b1cb4d6db1af5803ac376801`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:31.707461 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.825353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e7f1e-ac15-4299-954a-562b1a6cfc5b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:31.832705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:31.834328 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:31.834754 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:31.835370 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] PATH_INFO: `/v3/projects/11c9fdfdc9334939ac898e9b5ef6d35b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:31.837015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c9fdfdc9334939ac898e9b5ef6d35b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:31.840375 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:31.865043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1311/2629] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:31.888188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.890060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.925938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c9fdfdc9334939ac898e9b5ef6d35b', 'request_id': 'req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.935167 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:31.940038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c9fdfdc9334939ac898e9b5ef6d35b', 'request_id': 'req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.952737 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c9fdfdc9334939ac898e9b5ef6d35b', 'request_id': 'req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.974648 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] The token cache is being invalidate because project 11c9fdfdc9334939ac898e9b5ef6d35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:31.975151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.976251 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:31.979012 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1319/2630] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:31 2022] DELETE /identity/v3/projects/11c9fdfdc9334939ac898e9b5ef6d35b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:31.982318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:31.983025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:31.997639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd48ee-418f-48cb-aeb7-a718fc3e6245 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.093424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc87e6-5c54-41bd-af50-21178accd8d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.094742 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc87e6-5c54-41bd-af50-21178accd8d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.094955 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:32.095169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc87e6-5c54-41bd-af50-21178accd8d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.095475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc87e6-5c54-41bd-af50-21178accd8d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.099164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d9fc87e6-5c54-41bd-af50-21178accd8d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.105741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b679e-7be2-45da-9815-9f0d0f42a3f4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.115854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.117259 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.117679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.118148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] PATH_INFO: `/v3/projects/f1832985606d4180930387dd35cb789d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.120448 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1832985606d4180930387dd35cb789d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.124182 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.194774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1832985606d4180930387dd35cb789d', 'request_id': 'req-651ba15c-b088-4fcd-a37e-02eb700cf339'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.209192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1832985606d4180930387dd35cb789d', 'request_id': 'req-651ba15c-b088-4fcd-a37e-02eb700cf339'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.214036 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc87e6-5c54-41bd-af50-21178accd8d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.218770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc87e6-5c54-41bd-af50-21178accd8d6 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.231760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1832985606d4180930387dd35cb789d', 'request_id': 'req-651ba15c-b088-4fcd-a37e-02eb700cf339'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.232268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1312/2631] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:32.234754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe8b9b-73b3-4a97-b40d-f30aefcb84cf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.235201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe8b9b-73b3-4a97-b40d-f30aefcb84cf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.235659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe8b9b-73b3-4a97-b40d-f30aefcb84cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.259142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] The token cache is being invalidate because project f1832985606d4180930387dd35cb789d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:32.259641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.260865 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.263419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1320/2632] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:31 2022] DELETE /identity/v3/projects/f1832985606d4180930387dd35cb789d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:32.266630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9cbe8b9b-73b3-4a97-b40d-f30aefcb84cf None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:32.267588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.268761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9cbe8b9b-73b3-4a97-b40d-f30aefcb84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.269234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.384119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1313/2633] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:32.392808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fec59-b89a-4a6c-b25b-6c4fba159e85 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.393136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fec59-b89a-4a6c-b25b-6c4fba159e85 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.393380 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fec59-b89a-4a6c-b25b-6c4fba159e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.400355 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ba15c-b088-4fcd-a37e-02eb700cf339 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.408911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.411192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.411640 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.417043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] PATH_INFO: `/v3/users/8fc67261978244aeae990718ebded99e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.417043 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc67261978244aeae990718ebded99e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.417429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-601fec59-b89a-4a6c-b25b-6c4fba159e85 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:32.417429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-601fec59-b89a-4a6c-b25b-6c4fba159e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.422657 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.426610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1314/2634] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:32.433247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fec59-b89a-4a6c-b25b-6c4fba159e85 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.450574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fec59-b89a-4a6c-b25b-6c4fba159e85 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.457830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.458887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.459302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.459783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.461056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:32.461619 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.464648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.483513 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1315/2635] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:32.489195 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.495320 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:32.502740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b2dc5-5754-492a-86f7-8675592e2ce2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.508753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.509435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.509717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.510041 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.510862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.513028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.515649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc67261978244aeae990718ebded99e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.529457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc67261978244aeae990718ebded99e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.553281 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc67261978244aeae990718ebded99e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.554485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc67261978244aeae990718ebded99e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.571373 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1316/2636] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/projects => generated 359 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:32.576726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc67261978244aeae990718ebded99e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:32.579146 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.583636 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.588970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1321/2637] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:32 2022] DELETE /identity/v3/users/8fc67261978244aeae990718ebded99e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:32.727985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624d1ac-f205-4066-a55d-f43d37ad034a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.786773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.788813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:32.799658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:32.863045 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.863955 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.864389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.864814 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.866279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.870615 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.908064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a9a6f-606c-445c-96a1-7c4cdae9ff04 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.912868 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1317/2638] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/users => generated 280 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:32.917331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '853b4db3b1cb4d6db1af5803ac376801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb370de8c0c240fe9202df5230c65e49', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.917635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.918084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.918474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.919325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.920128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:32.920477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.922382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.941384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa4803-27be-4346-8176-df81812a35a0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.945908 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1322/2639] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:32.947965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:32.953228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.954367 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.954850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.955311 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.957094 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.959772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.966562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a1979-d237-48f4-865f-b3ba244ca131 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:32.974869 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '853b4db3b1cb4d6db1af5803ac376801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb370de8c0c240fe9202df5230c65e49', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:32.976483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:32.976483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:32.976864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:32.977976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:32.978857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1318/2640] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:32.980502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:32.983622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.002850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2bbc0-08a9-44a9-8a64-5797bef8a744 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.009535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.010662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.011118 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.011548 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.012903 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.015538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.037966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1323/2641] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:32 2022] POST /identity/v3/projects => generated 354 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:33.039230 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.044949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1319/2642] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:33.052049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.071286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c95b5-6e7b-43ee-ae7f-7731490a53fd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.160066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21275c13-d1cb-4254-8991-9a85996655f6 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.165621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '853b4db3b1cb4d6db1af5803ac376801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb370de8c0c240fe9202df5230c65e49', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.166348 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.166645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.166931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.167692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:33.168007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.169643 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.175950 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:33.186200 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1324/2643] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:33.188409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.189081 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.190259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.190627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.192007 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.194173 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] PATH_INFO: `/v3/projects/f2faf28db3b8415da4748d1fa6297230/users/b24d80e38214485dac52169da23de317/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.222057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2faf28db3b8415da4748d1fa6297230, user_id=b24d80e38214485dac52169da23de317, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.222373 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6630d94-ef5f-42bb-893b-aa12ee43ad84 tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.226542 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.230870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.233404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.234237 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.234612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] PATH_INFO: `/v3/projects/839cc5c34b2e48138226d7bd75f0ac89` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.236997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=839cc5c34b2e48138226d7bd75f0ac89)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.241796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.253921 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1320/2644] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:33 2022] PUT /identity/v3/projects/f2faf28db3b8415da4748d1fa6297230/users/b24d80e38214485dac52169da23de317/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:33.255805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.331529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839cc5c34b2e48138226d7bd75f0ac89', 'request_id': 'req-8e927ce4-347f-428e-8918-e81a35c59fbe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.344317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839cc5c34b2e48138226d7bd75f0ac89', 'request_id': 'req-8e927ce4-347f-428e-8918-e81a35c59fbe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.354619 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839cc5c34b2e48138226d7bd75f0ac89', 'request_id': 'req-8e927ce4-347f-428e-8918-e81a35c59fbe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.356770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:33.371793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab609dc1-ba21-40eb-b89f-a88c99c0d170 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.382618 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] The token cache is being invalidate because project 839cc5c34b2e48138226d7bd75f0ac89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:33.383395 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.385036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e927ce4-347f-428e-8918-e81a35c59fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.387871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1325/2645] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:33 2022] DELETE /identity/v3/projects/839cc5c34b2e48138226d7bd75f0ac89 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:33.390567 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fff749-66af-482c-bd00-75b19d4a7982 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.391080 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fff749-66af-482c-bd00-75b19d4a7982 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.391682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fff749-66af-482c-bd00-75b19d4a7982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.441277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-24fff749-66af-482c-bd00-75b19d4a7982 None None] MFA Rules not processed for user `b24d80e38214485dac52169da23de317`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:33.443523 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-24fff749-66af-482c-bd00-75b19d4a7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.451893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '853b4db3b1cb4d6db1af5803ac376801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cb370de8c0c240fe9202df5230c65e49', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.452754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.453192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.453471 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] PATH_INFO: `/v3/projects/ba1f4ee4f7064d2fac2f062587028a4a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.454734 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ba1f4ee4f7064d2fac2f062587028a4a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.456678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.538128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1f4ee4f7064d2fac2f062587028a4a', 'request_id': 'req-49f3ddb6-cb0f-4412-8d51-d336366ba42d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.549250 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1f4ee4f7064d2fac2f062587028a4a', 'request_id': 'req-49f3ddb6-cb0f-4412-8d51-d336366ba42d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.561439 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1326/2646] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:33.562655 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1f4ee4f7064d2fac2f062587028a4a', 'request_id': 'req-49f3ddb6-cb0f-4412-8d51-d336366ba42d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.568611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b2ab0-bb24-4ade-9217-8e80b0509f0f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.569345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b2ab0-bb24-4ade-9217-8e80b0509f0f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.569345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b2ab0-bb24-4ade-9217-8e80b0509f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.586337 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] The token cache is being invalidate because project ba1f4ee4f7064d2fac2f062587028a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:33.586958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.588541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:33.590915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1321/2647] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:33 2022] DELETE /identity/v3/projects/ba1f4ee4f7064d2fac2f062587028a4a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:33.599957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-012b2ab0-bb24-4ade-9217-8e80b0509f0f None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:33.602263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-012b2ab0-bb24-4ade-9217-8e80b0509f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.609374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.611399 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.646052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:33.717762 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3ddb6-cb0f-4412-8d51-d336366ba42d tempest-VolumeQuotaClassesTest-2021297361 tempest-VolumeQuotaClassesTest-2021297361-project-admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.723261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1327/2648] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:33.725987 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.727124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.727616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.727956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.730593 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.735708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0aa6e-20fe-4e58-b396-eba7c4a54a64 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.736176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0aa6e-20fe-4e58-b396-eba7c4a54a64 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.736622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0aa6e-20fe-4e58-b396-eba7c4a54a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.763726 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c1a0aa6e-20fe-4e58-b396-eba7c4a54a64 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:33.765130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c1a0aa6e-20fe-4e58-b396-eba7c4a54a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.775629 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1328/2649] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:33.781148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0aa6e-20fe-4e58-b396-eba7c4a54a64 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.798078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0aa6e-20fe-4e58-b396-eba7c4a54a64 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.816525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.818113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.818707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.819322 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.820719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:33.821518 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.824792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.831712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.833221 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.844104 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1322/2650] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:33 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:33.846780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1329/2651] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:33.851906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.866985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.869601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:33.881105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bade68-4154-4581-959e-c7ce8ee49967 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.888809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.889893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.890355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.890855 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.892028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.892841 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc393a89-e754-453a-9798-d7df6d32fbcc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:33.895714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.902844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:33.904666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:33.905316 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:33.905813 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] PATH_INFO: `/v3/users/b24d80e38214485dac52169da23de317` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:33.908645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24d80e38214485dac52169da23de317)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:33.911462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:33.945048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1323/2652] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:33.954192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:33.990715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24d80e38214485dac52169da23de317'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.005313 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24d80e38214485dac52169da23de317'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.011076 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:34.018895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24d80e38214485dac52169da23de317'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.019976 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24d80e38214485dac52169da23de317'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.048765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24d80e38214485dac52169da23de317'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.058184 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05be5d-f8f4-461a-92d1-24b8d6282c4b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.059090 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1330/2653] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:33 2022] DELETE /identity/v3/users/b24d80e38214485dac52169da23de317 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:34.060148 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.061661 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.142817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:34.162151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec39e9-89ec-480f-9c14-0506251b07b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.163453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec39e9-89ec-480f-9c14-0506251b07b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.164161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec39e9-89ec-480f-9c14-0506251b07b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.164700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec39e9-89ec-480f-9c14-0506251b07b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.166056 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec39e9-89ec-480f-9c14-0506251b07b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.169906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec39e9-89ec-480f-9c14-0506251b07b2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.178323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09128054-3d12-4fc9-ad74-cf920da62dc0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.185927 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.187452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.187945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.188449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] PATH_INFO: `/v3/users/f8418981b69d491da8c5de490b71f898` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.191225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8418981b69d491da8c5de490b71f898)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.194633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.205527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1324/2654] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:33 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:34.207947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.208451 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.208939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.246264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:34.248353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.258875 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1325/2655] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:34.259857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.261328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.279736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc856b59-dda4-49f4-853b-faa8286ba649 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.283895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8418981b69d491da8c5de490b71f898'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.289204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.290425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.290935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.291441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.292826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.297916 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.298399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8418981b69d491da8c5de490b71f898'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.312215 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8418981b69d491da8c5de490b71f898'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.313483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8418981b69d491da8c5de490b71f898'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.319007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1326/2656] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:34.319007 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.323077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.335854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8418981b69d491da8c5de490b71f898'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:34.346146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1331/2657] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:34 2022] DELETE /identity/v3/users/f8418981b69d491da8c5de490b71f898 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:34.348202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.349872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.427967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:34.430334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4087aa-50dd-4287-a7c0-d376dfbb4705 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.435535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e4e1-2841-45d8-bd10-381e1968ea52 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.443176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c8eff-82f9-4143-abd8-49b952af250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.443868 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c8eff-82f9-4143-abd8-49b952af250a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.444155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c8eff-82f9-4143-abd8-49b952af250a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.444465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c8eff-82f9-4143-abd8-49b952af250a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.445209 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c8eff-82f9-4143-abd8-49b952af250a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.446921 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c8eff-82f9-4143-abd8-49b952af250a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.459905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1332/2658] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:34.462023 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed8c50-1840-4759-af39-2e5b957927d5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.462691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed8c50-1840-4759-af39-2e5b957927d5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.463176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed8c50-1840-4759-af39-2e5b957927d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.484163 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f4ed8c50-1840-4759-af39-2e5b957927d5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:34.485482 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f4ed8c50-1840-4759-af39-2e5b957927d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.496450 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1333/2659] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:34.497671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed8c50-1840-4759-af39-2e5b957927d5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.505739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.507627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.508237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.508858 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.512508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.513599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed8c50-1840-4759-af39-2e5b957927d5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.521800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90fd71-c471-49bd-94e8-b84e5b5a5cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.523347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fd71-c471-49bd-94e8-b84e5b5a5cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.523837 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fd71-c471-49bd-94e8-b84e5b5a5cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.524324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90fd71-c471-49bd-94e8-b84e5b5a5cea admin admin] PATH_INFO: `/v3/projects/2884447e2332431bad43869dd9af8bb2/users/77b1c2dca3fa4d57a5258fabeeac6cdd/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.531248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.533584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.549441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1327/2660] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:34.552209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.559081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90fd71-c471-49bd-94e8-b84e5b5a5cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2884447e2332431bad43869dd9af8bb2, user_id=77b1c2dca3fa4d57a5258fabeeac6cdd, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.563719 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d4771-d8cb-4ddb-bca8-3feff6eca86f service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.565150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90fd71-c471-49bd-94e8-b84e5b5a5cea admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.574085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.574085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.574085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.574085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.588524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1334/2661] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:34 2022] PUT /identity/v3/projects/2884447e2332431bad43869dd9af8bb2/users/77b1c2dca3fa4d57a5258fabeeac6cdd/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:34.591491 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.594765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.600398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46dc2-6455-4848-a96f-60c2d3a51c51 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.600759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46dc2-6455-4848-a96f-60c2d3a51c51 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.601015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46dc2-6455-4848-a96f-60c2d3a51c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.616787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1328/2662] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:34.641028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-88d46dc2-6455-4848-a96f-60c2d3a51c51 None None] MFA Rules not processed for user `77b1c2dca3fa4d57a5258fabeeac6cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:34.643018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-88d46dc2-6455-4848-a96f-60c2d3a51c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:34.762268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1335/2663] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:34.768159 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.781364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e354b-84ae-4c43-8621-b1321322628b service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:34.893673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:34.895142 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:34.895772 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:34.896417 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:34.897888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:34.902400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:34.947957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:34.964972 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1329/2664] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:34 2022] POST /identity/v3/projects => generated 363 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:34.971205 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d46dc2-6455-4848-a96f-60c2d3a51c51 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:34.992422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d46dc2-6455-4848-a96f-60c2d3a51c51 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.096408 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.098280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.098800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.099508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.100825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.104715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.148123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.152611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1336/2665] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:34 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:35.154892 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.155322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:35.174693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a0195-6f98-4b6e-92be-c9595edd2bc1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.183434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.184139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.184427 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.184703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.184820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8e99-39a9-4c69-812f-35749daf6140 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.185449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.187138 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.192416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.194106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.194404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.194691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] PATH_INFO: `/v3/projects/f2faf28db3b8415da4748d1fa6297230` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.196547 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2faf28db3b8415da4748d1fa6297230)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.200853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.214157 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1337/2666] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:35.215771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.220418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:35.244975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aa327-086d-4bbe-bd9c-09cf764ff3a1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.252660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.253601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.253909 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.254195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] PATH_INFO: `/v3/projects/6d7dd56eefed455f807a41990f358efe` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.255765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d7dd56eefed455f807a41990f358efe)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.257876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.278458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2faf28db3b8415da4748d1fa6297230', 'request_id': 'req-0c68eaca-614a-4cd8-9016-384ea6bb51a5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.291043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2faf28db3b8415da4748d1fa6297230', 'request_id': 'req-0c68eaca-614a-4cd8-9016-384ea6bb51a5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.294884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:35.305116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2faf28db3b8415da4748d1fa6297230', 'request_id': 'req-0c68eaca-614a-4cd8-9016-384ea6bb51a5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.325566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] The token cache is being invalidate because project f2faf28db3b8415da4748d1fa6297230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:35.326161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.327394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.330040 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1330/2667] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:35 2022] DELETE /identity/v3/projects/f2faf28db3b8415da4748d1fa6297230 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:35.334424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.334424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:35.335953 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d7dd56eefed455f807a41990f358efe', 'request_id': 'req-69a91db3-86a6-4a5c-a304-c462c0427ae9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.344992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d7dd56eefed455f807a41990f358efe', 'request_id': 'req-69a91db3-86a6-4a5c-a304-c462c0427ae9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.352828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d7dd56eefed455f807a41990f358efe', 'request_id': 'req-69a91db3-86a6-4a5c-a304-c462c0427ae9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.367377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] The token cache is being invalidate because project 6d7dd56eefed455f807a41990f358efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:35.367919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.368881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69a91db3-86a6-4a5c-a304-c462c0427ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.371811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1338/2668] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:35 2022] DELETE /identity/v3/projects/6d7dd56eefed455f807a41990f358efe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:35.374195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.374691 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.375390 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.406739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:35.409194 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:35.430366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68eaca-614a-4cd8-9016-384ea6bb51a5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.510496 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1339/2669] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:35.511548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.513009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:35.527580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa909-5f5f-401f-b161-cdd8845db44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.527940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b471c98-1c2f-4476-9544-50829ea7edde None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.529162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa909-5f5f-401f-b161-cdd8845db44a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.529663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa909-5f5f-401f-b161-cdd8845db44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.530382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa909-5f5f-401f-b161-cdd8845db44a admin admin] PATH_INFO: `/v3/projects/a9442801ef474fe1a7fc266c49129efa/users/dd2385deb64042c6aa343fbbc2aabde9/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.533188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.537161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.537475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.537793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] PATH_INFO: `/v3/users/853b4db3b1cb4d6db1af5803ac376801` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.539244 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=853b4db3b1cb4d6db1af5803ac376801)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.541221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.558643 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa909-5f5f-401f-b161-cdd8845db44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9442801ef474fe1a7fc266c49129efa, user_id=dd2385deb64042c6aa343fbbc2aabde9, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.562424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa909-5f5f-401f-b161-cdd8845db44a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.579793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1331/2670] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:35 2022] PUT /identity/v3/projects/a9442801ef474fe1a7fc266c49129efa/users/dd2385deb64042c6aa343fbbc2aabde9/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:35.583614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9c88-d888-4fc5-a0a3-b21cfed3e9c5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.584099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9c88-d888-4fc5-a0a3-b21cfed3e9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.584572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f9c88-d888-4fc5-a0a3-b21cfed3e9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.612404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-817f9c88-d888-4fc5-a0a3-b21cfed3e9c5 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:35.615126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-817f9c88-d888-4fc5-a0a3-b21cfed3e9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:35.631746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:35.634986 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853b4db3b1cb4d6db1af5803ac376801'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.648175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853b4db3b1cb4d6db1af5803ac376801'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.658247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853b4db3b1cb4d6db1af5803ac376801'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.659820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853b4db3b1cb4d6db1af5803ac376801'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.683298 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853b4db3b1cb4d6db1af5803ac376801'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:35.695207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1340/2671] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:35 2022] DELETE /identity/v3/users/853b4db3b1cb4d6db1af5803ac376801 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:35.696340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.698609 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:35.723390 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1332/2672] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:35.728411 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f9c88-d888-4fc5-a0a3-b21cfed3e9c5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.744695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f9c88-d888-4fc5-a0a3-b21cfed3e9c5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.759277 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:35.846053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29146304-caaa-4092-9128-9af5605b51ea admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.846579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.847652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.848087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.848639 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.849722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:35.850156 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.852659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.855676 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.857151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.857765 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.858402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.859822 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.863158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.872090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1333/2673] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:35.876637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.880550 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1341/2674] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:35.885257 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:35.895236 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeda64-a92f-4555-85e1-165202223bbc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.909015 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.910209 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.910695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.911185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.912457 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.915558 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9216d8-8a3f-40a5-89b1-a6444685ceeb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:35.915980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.925262 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53567f5e-2c69-4ae0-a9db-320e93bf4316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:35.926972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567f5e-2c69-4ae0-a9db-320e93bf4316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.927572 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567f5e-2c69-4ae0-a9db-320e93bf4316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.928186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53567f5e-2c69-4ae0-a9db-320e93bf4316 admin admin] PATH_INFO: `/v3/projects/a9442801ef474fe1a7fc266c49129efa/users/dd2385deb64042c6aa343fbbc2aabde9/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.931867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53567f5e-2c69-4ae0-a9db-320e93bf4316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9442801ef474fe1a7fc266c49129efa, user_id=dd2385deb64042c6aa343fbbc2aabde9, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:35.937881 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53567f5e-2c69-4ae0-a9db-320e93bf4316 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:35.962118 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1342/2675] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:35 2022] PUT /identity/v3/projects/a9442801ef474fe1a7fc266c49129efa/users/dd2385deb64042c6aa343fbbc2aabde9/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:35.976916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:35.977590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:35.978508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:35.978663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1334/2676] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:35 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:35.983712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.029863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] MFA Rules not processed for user `dd2385deb64042c6aa343fbbc2aabde9`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:36.032137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.107907 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7969c-a4d9-44cc-9802-a02b14f34c70 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.117699 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.118931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.119243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.119530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.120652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.124017 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.158818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1343/2677] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:35 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:36.160413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.162538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.164578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1335/2678] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:35 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:36.168781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.183532 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be40f8-e46a-4546-9964-d89fae99179b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.185016 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e43ae-5fb0-4210-ace3-093d814e3945 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.191421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.192005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.192773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.192946 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.193181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.193310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.193579 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.193801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] PATH_INFO: `/v3/projects/cb370de8c0c240fe9202df5230c65e49` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.194514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.195896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb370de8c0c240fe9202df5230c65e49)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.196295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.199612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.214937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1336/2679] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:36.214937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.214937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.220006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:36.273123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb370de8c0c240fe9202df5230c65e49', 'request_id': 'req-39751d67-dcf3-4bf7-b5ab-b1744418917a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:36.286951 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb370de8c0c240fe9202df5230c65e49', 'request_id': 'req-39751d67-dcf3-4bf7-b5ab-b1744418917a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:36.300173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb370de8c0c240fe9202df5230c65e49', 'request_id': 'req-39751d67-dcf3-4bf7-b5ab-b1744418917a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:36.307528 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d0182-465f-434d-ac66-98fc299a3202 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.317093 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] The token cache is being invalidate because project cb370de8c0c240fe9202df5230c65e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:36.317391 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:36.318119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:36.319895 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1344/2680] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:36 2022] DELETE /identity/v3/projects/cb370de8c0c240fe9202df5230c65e49 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:36.325376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.325376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.344520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:36.375604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.376170 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.376453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.376736 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.378717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.430327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39751d67-dcf3-4bf7-b5ab-b1744418917a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.438500 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e0848-f185-489c-bcec-1cdb08ce9197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.439199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e0848-f185-489c-bcec-1cdb08ce9197 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.439488 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e0848-f185-489c-bcec-1cdb08ce9197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.439770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e0848-f185-489c-bcec-1cdb08ce9197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.441070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e0848-f185-489c-bcec-1cdb08ce9197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.442922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e0848-f185-489c-bcec-1cdb08ce9197 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.455864 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1345/2681] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:36.457602 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e5f9b-134d-4597-9125-6fc91e7eb550 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.457900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e5f9b-134d-4597-9125-6fc91e7eb550 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.458190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e5f9b-134d-4597-9125-6fc91e7eb550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.468120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.470282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.481057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3d1e5f9b-134d-4597-9125-6fc91e7eb550 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:36.482345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3d1e5f9b-134d-4597-9125-6fc91e7eb550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.483874 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1337/2682] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:36.485301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.491289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1346/2683] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:36.501773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b1ca6-1ee4-444c-9861-fc6dd48783d8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.502147 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b1ca6-1ee4-444c-9861-fc6dd48783d8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.502514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b1ca6-1ee4-444c-9861-fc6dd48783d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.506445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b69d4d-68ce-4f65-9416-cc9de492f278 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.514694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed8db1-07a5-4d64-8d08-13724221d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.516190 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed8db1-07a5-4d64-8d08-13724221d167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.516662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed8db1-07a5-4d64-8d08-13724221d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.517445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed8db1-07a5-4d64-8d08-13724221d167 admin admin] PATH_INFO: `/v3/projects/f8ba2c3616c24a408c4003277db19262/users/0e5ef3cd501e4baa8e6ac5f2574531fb/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.529275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2d2b1ca6-1ee4-444c-9861-fc6dd48783d8 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:36.531580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2d2b1ca6-1ee4-444c-9861-fc6dd48783d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.542738 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed8db1-07a5-4d64-8d08-13724221d167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ba2c3616c24a408c4003277db19262, user_id=0e5ef3cd501e4baa8e6ac5f2574531fb, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.543744 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1347/2684] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:36.545725 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed8db1-07a5-4d64-8d08-13724221d167 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.548229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b1ca6-1ee4-444c-9861-fc6dd48783d8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.563536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1338/2685] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:36 2022] PUT /identity/v3/projects/f8ba2c3616c24a408c4003277db19262/users/0e5ef3cd501e4baa8e6ac5f2574531fb/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:36.570003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b1ca6-1ee4-444c-9861-fc6dd48783d8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.571624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17a31-9fac-4bdc-8f1a-8e1cf5e04433 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.572090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17a31-9fac-4bdc-8f1a-8e1cf5e04433 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.572578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17a31-9fac-4bdc-8f1a-8e1cf5e04433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.607033 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-20f17a31-9fac-4bdc-8f1a-8e1cf5e04433 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:36.608724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-20f17a31-9fac-4bdc-8f1a-8e1cf5e04433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.661190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.662195 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.662484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.662763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.663690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:36.665000 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.666593 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.679937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:36.680340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1348/2686] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:36.681614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.683794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.708813 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1339/2687] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:36.710058 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17a31-9fac-4bdc-8f1a-8e1cf5e04433 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.726771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17a31-9fac-4bdc-8f1a-8e1cf5e04433 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.734841 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.735969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.736475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.736970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.738339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.741991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.764106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8b4c1-11d2-4714-a4bf-005d4b95e1f7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.771074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.772425 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.772780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.773091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.776481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.787908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1340/2688] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:36 2022] POST /identity/v3/projects => generated 389 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:36.789133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.791358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:36.838928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:36.859693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:36.862825 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:36.876269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51c69d-c54c-4893-ac43-faf1335ea697 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.879396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1349/2689] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:36.880685 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:36.885609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.886674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.886977 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.887263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.899095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab97fb4-de57-4d96-90f3-e0eae8b1002a service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:36.995898 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:36.997285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:36.997730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:36.998247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:36.999494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.002714 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.005131 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.007887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.024343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1341/2690] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:37.036544 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1350/2691] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:36 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:37.043406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.066220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d5984-45f9-42e2-9e18-fba1abbff181 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:37.075991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:37.077259 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.077937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.078408 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.079826 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.083348 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.106708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1342/2692] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:37.111670 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.124056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7b810-4cc4-4216-8487-e42207998fea admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:37.136580 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8bcc5-99de-4fb6-a395-949a58eaa359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:37.137761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8bcc5-99de-4fb6-a395-949a58eaa359 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.138481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8bcc5-99de-4fb6-a395-949a58eaa359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.138687 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8bcc5-99de-4fb6-a395-949a58eaa359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.139925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8bcc5-99de-4fb6-a395-949a58eaa359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.143392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8bcc5-99de-4fb6-a395-949a58eaa359 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.162885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1351/2693] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:37.168531 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.182086 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb450a-da1d-4ae0-a9c6-452b999dadf9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:37.189369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:37.191339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.191857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.192493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] PATH_INFO: `/v3/projects/2ba09001a90a4656869d58953cf8820d/users/280f2a4a09224904b5dabb8f8238769c/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.222583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba09001a90a4656869d58953cf8820d, user_id=280f2a4a09224904b5dabb8f8238769c, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.227877 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.253243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1343/2694] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:37 2022] PUT /identity/v3/projects/2ba09001a90a4656869d58953cf8820d/users/280f2a4a09224904b5dabb8f8238769c/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:37.271398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74147ee-4117-4441-9f95-a84e5c4a0b5b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.271734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74147ee-4117-4441-9f95-a84e5c4a0b5b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.271969 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74147ee-4117-4441-9f95-a84e5c4a0b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.317263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d74147ee-4117-4441-9f95-a84e5c4a0b5b None None] MFA Rules not processed for user `280f2a4a09224904b5dabb8f8238769c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:37.318601 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d74147ee-4117-4441-9f95-a84e5c4a0b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:37.403730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1352/2695] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:37.411242 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.505395 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:37.550849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1e67-9e1e-4d3c-8246-27ad95162bf2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:37.558687 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:37.560421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.560421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.560895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.562243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.566743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.608860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1344/2696] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:37 2022] POST /identity/v3/projects => generated 389 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:37.610261 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74147ee-4117-4441-9f95-a84e5c4a0b5b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.624735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74147ee-4117-4441-9f95-a84e5c4a0b5b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:37.703228 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:37.715170 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:37.716596 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.717125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.717798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.719208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.726117 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.760819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1353/2697] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:37 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:37.766092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.780583 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088386f-5517-4e6c-b6c2-80c766fdece3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:37.787135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:37.788315 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.788852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.789183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.791539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.795148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.813799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1345/2698] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:37.819919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.839022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51a805-1b39-4953-8d60-01a1fd37412b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:37.846374 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:37.847887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:37.848525 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:37.848870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] PATH_INFO: `/v3/projects/7187828e9e1349af9c69789eb5867a73/users/3a681e84589c4a0b9851fa7c78e01ef3/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:37.875954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7187828e9e1349af9c69789eb5867a73, user_id=3a681e84589c4a0b9851fa7c78e01ef3, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:37.882801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:37.909961 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1354/2699] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:37 2022] PUT /identity/v3/projects/7187828e9e1349af9c69789eb5867a73/users/3a681e84589c4a0b9851fa7c78e01ef3/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:37.916330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:37.940151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4119c4c-4b2d-42a5-ba4e-ae92b0eef5de admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:38.055084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ded82-129f-46c0-9068-eae21a3f6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:38.056974 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ded82-129f-46c0-9068-eae21a3f6680 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:38.057651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ded82-129f-46c0-9068-eae21a3f6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:38.058476 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ded82-129f-46c0-9068-eae21a3f6680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:38.060114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ded82-129f-46c0-9068-eae21a3f6680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:38.064235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ded82-129f-46c0-9068-eae21a3f6680 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:38.085506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1346/2700] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:38.091067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:38.104067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c58a-46d4-4faa-9707-bef5bb1aeed7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:38.111707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:38.113882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:38.113882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:38.114256 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] PATH_INFO: `/v3/projects/7187828e9e1349af9c69789eb5867a73/users/3a681e84589c4a0b9851fa7c78e01ef3/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:38.117268 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7187828e9e1349af9c69789eb5867a73, user_id=3a681e84589c4a0b9851fa7c78e01ef3, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:38.120463 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:38.139850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1355/2701] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:38 2022] PUT /identity/v3/projects/7187828e9e1349af9c69789eb5867a73/users/3a681e84589c4a0b9851fa7c78e01ef3/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:38.158674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1454b80-6c35-48f2-b224-c8332b3a73ed None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:38.160604 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1454b80-6c35-48f2-b224-c8332b3a73ed None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:38.160793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1454b80-6c35-48f2-b224-c8332b3a73ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:38.226514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d1454b80-6c35-48f2-b224-c8332b3a73ed None None] MFA Rules not processed for user `3a681e84589c4a0b9851fa7c78e01ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:38.228915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1454b80-6c35-48f2-b224-c8332b3a73ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:38.377757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1347/2702] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:38.394303 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:38.508559 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daeebb-fd59-4bf8-8709-0760d93c16f6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:38.514053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137bb43-7a07-4178-9016-6cd0f9469cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:38.514917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137bb43-7a07-4178-9016-6cd0f9469cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:38.515203 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137bb43-7a07-4178-9016-6cd0f9469cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:38.515564 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137bb43-7a07-4178-9016-6cd0f9469cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:38.530696 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137bb43-7a07-4178-9016-6cd0f9469cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:38.534940 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137bb43-7a07-4178-9016-6cd0f9469cf2 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:38.549761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1356/2703] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:38.780235 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1454b80-6c35-48f2-b224-c8332b3a73ed None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:38.843521 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:38.911439 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1454b80-6c35-48f2-b224-c8332b3a73ed None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:38.926552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:38.927760 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:38.928122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:38.928514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] PATH_INFO: `/v3/users/280f2a4a09224904b5dabb8f8238769c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:38.929993 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=280f2a4a09224904b5dabb8f8238769c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:38.931998 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:39.004262 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280f2a4a09224904b5dabb8f8238769c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:39.018303 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280f2a4a09224904b5dabb8f8238769c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:39.042147 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280f2a4a09224904b5dabb8f8238769c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:39.043286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280f2a4a09224904b5dabb8f8238769c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:39.068224 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280f2a4a09224904b5dabb8f8238769c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:39.084355 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1348/2704] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:38 2022] DELETE /identity/v3/users/280f2a4a09224904b5dabb8f8238769c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:39.090022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:39.090911 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:39.091524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:39.133492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:39.135143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:39.242078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1357/2705] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:39.253718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:39.255904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:39.421216 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2e7b5-27f4-4492-9e53-ed5a1194b0bb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:39.426514 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:39.427207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:39.427670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:39.427964 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:39.439853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:39.442006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:39.457928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1349/2706] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:39.844449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:39.847132 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:39.863806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eb0d0-2a2d-4c8b-b3f9-e47b28ed2f0e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:39.869889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:39.871036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:39.871239 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:39.871720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] PATH_INFO: `/v3/projects/2ba09001a90a4656869d58953cf8820d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:39.875880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba09001a90a4656869d58953cf8820d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:39.881176 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:39.987468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:39.994356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba09001a90a4656869d58953cf8820d', 'request_id': 'req-2bf4687f-ecf4-4908-805d-7051a313ae3e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.009671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba09001a90a4656869d58953cf8820d', 'request_id': 'req-2bf4687f-ecf4-4908-805d-7051a313ae3e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.027223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba09001a90a4656869d58953cf8820d', 'request_id': 'req-2bf4687f-ecf4-4908-805d-7051a313ae3e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.054139 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] The token cache is being invalidate because project 2ba09001a90a4656869d58953cf8820d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:40.054735 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.056547 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.062800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1358/2707] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:39 2022] DELETE /identity/v3/projects/2ba09001a90a4656869d58953cf8820d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:40.064648 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:40.084572 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9cf72-38ce-43a2-8907-f31d184443c4 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:40.258434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:40.282673 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:40.289928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:40.289928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:40.289928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] PATH_INFO: `/v3/users/3a681e84589c4a0b9851fa7c78e01ef3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:40.292124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a681e84589c4a0b9851fa7c78e01ef3)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:40.294163 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:40.387253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a681e84589c4a0b9851fa7c78e01ef3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.399804 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a681e84589c4a0b9851fa7c78e01ef3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.418674 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a681e84589c4a0b9851fa7c78e01ef3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.419181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a681e84589c4a0b9851fa7c78e01ef3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.452827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e8fa91f-4cf6-4f0c-9d11-a215341373eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a681e84589c4a0b9851fa7c78e01ef3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:40.469018 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1350/2708] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:40 2022] DELETE /identity/v3/users/3a681e84589c4a0b9851fa7c78e01ef3 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:41.172520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:41.174497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:41.195907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4687f-ecf4-4908-805d-7051a313ae3e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:41.395917 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:41.397114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:41.397570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:41.398538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] PATH_INFO: `/v3/projects/7187828e9e1349af9c69789eb5867a73` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:41.400027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7187828e9e1349af9c69789eb5867a73)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:41.402710 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:41.495049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7187828e9e1349af9c69789eb5867a73', 'request_id': 'req-cabb0d0f-507b-4784-ac0b-b754714a2550'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:41.509334 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7187828e9e1349af9c69789eb5867a73', 'request_id': 'req-cabb0d0f-507b-4784-ac0b-b754714a2550'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:41.521340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7187828e9e1349af9c69789eb5867a73', 'request_id': 'req-cabb0d0f-507b-4784-ac0b-b754714a2550'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:41.560325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] The token cache is being invalidate because project 7187828e9e1349af9c69789eb5867a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:41.562302 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:41.563651 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cabb0d0f-507b-4784-ac0b-b754714a2550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:41.567461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1359/2709] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:41 2022] DELETE /identity/v3/projects/7187828e9e1349af9c69789eb5867a73 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:41.580863 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197238-b965-4616-9a1e-8403826590c3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:41.581443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197238-b965-4616-9a1e-8403826590c3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:41.582313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5197238-b965-4616-9a1e-8403826590c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:41.653185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a5197238-b965-4616-9a1e-8403826590c3 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:41.655458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a5197238-b965-4616-9a1e-8403826590c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:41.778339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1351/2710] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:41.788497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3a25f-83c7-4dfb-9b4f-364cae96cb26 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:41.788819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3a25f-83c7-4dfb-9b4f-364cae96cb26 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:41.789051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3a25f-83c7-4dfb-9b4f-364cae96cb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:41.810069 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-05c3a25f-83c7-4dfb-9b4f-364cae96cb26 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:41.811773 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-05c3a25f-83c7-4dfb-9b4f-364cae96cb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:41.824465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1360/2711] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:41.828382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5197238-b965-4616-9a1e-8403826590c3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:41.847075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5197238-b965-4616-9a1e-8403826590c3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:41.856345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:41.858656 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:41.859182 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:41.859698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:41.863107 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:41.863669 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:41.868342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:41.902819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1352/2712] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:48:41.906807 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3a25f-83c7-4dfb-9b4f-364cae96cb26 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:41.934337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3a25f-83c7-4dfb-9b4f-364cae96cb26 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:41.942789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:41.943937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:41.944437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:41.944938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:41.946295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:41.954799 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:42.007389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1361/2713] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:41 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:42.013586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:42.024584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16140-1633-465a-a1fd-51fb66165b6c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:42.130155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:42.130968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:42.131357 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:42.131755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:42.132614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:42.135446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:42.149662 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:42.178248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1353/2714] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:42 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:42.180084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:42.199566 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f67d-7161-4475-82a2-829adb71b5d1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:42.219731 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:42.221027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:42.222153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:42.226273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:42.227793 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:42.236119 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:42.273210 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1362/2715] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:42.278817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:42.294825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346fe86-ef31-4f44-b877-5defaed4f9b1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:42.309737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d29304-0587-4ab1-b7eb-c8541d80e6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:42.309737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29304-0587-4ab1-b7eb-c8541d80e6de admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:42.309737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29304-0587-4ab1-b7eb-c8541d80e6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:42.310587 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d29304-0587-4ab1-b7eb-c8541d80e6de admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:42.311911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29304-0587-4ab1-b7eb-c8541d80e6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:42.326168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d29304-0587-4ab1-b7eb-c8541d80e6de admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:42.380754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1354/2716] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:42.387301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:42.408695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5a863-17d1-41b5-a3bb-824d786a5487 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:42.424347 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:42.426357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:42.426897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:42.427394 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] PATH_INFO: `/v3/projects/4ccb8e176b714bb89d00adee9521890e/users/2834625eaa054138ba3afc3ef7f9214a/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:42.468557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccb8e176b714bb89d00adee9521890e, user_id=2834625eaa054138ba3afc3ef7f9214a, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:42.474665 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:42.504932 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:42.512379 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1363/2717] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:42 2022] PUT /identity/v3/projects/4ccb8e176b714bb89d00adee9521890e/users/2834625eaa054138ba3afc3ef7f9214a/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:42.523957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6dd0d-fb20-4248-82b2-a07a85da7161 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:42.525042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6dd0d-fb20-4248-82b2-a07a85da7161 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:42.526071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6dd0d-fb20-4248-82b2-a07a85da7161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:42.588128 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d3a6dd0d-fb20-4248-82b2-a07a85da7161 None None] MFA Rules not processed for user `2834625eaa054138ba3afc3ef7f9214a`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:42.589546 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d3a6dd0d-fb20-4248-82b2-a07a85da7161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:42.723893 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1355/2718] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:42.739081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:42.740942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:42.843005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c35b9a-683e-46af-88d2-e9be8be47caa admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:42.850871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:42.852922 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:42.854040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:42.854783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:42.872595 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:42.875984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:42.902325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1364/2719] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:43.555621 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e72d-13a5-4fc7-9eb2-b219da8f5d6d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:43.556316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e72d-13a5-4fc7-9eb2-b219da8f5d6d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:43.556678 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e72d-13a5-4fc7-9eb2-b219da8f5d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:43.619980 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2018e72d-13a5-4fc7-9eb2-b219da8f5d6d None None] MFA Rules not processed for user `0e5ef3cd501e4baa8e6ac5f2574531fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:43.621425 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2018e72d-13a5-4fc7-9eb2-b219da8f5d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:43.795919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1356/2720] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:43.797570 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:43.800484 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:43.890095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09413d-550b-4e10-98be-401b741e8d23 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:43.896761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:43.897935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:43.898280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:43.898628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:43.899563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:43.902623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:43.942957 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1365/2721] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:43 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:43.948727 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018e72d-13a5-4fc7-9eb2-b219da8f5d6d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:43.975857 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018e72d-13a5-4fc7-9eb2-b219da8f5d6d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:44.055351 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:44.056106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:44.056580 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:44.056872 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:44.057601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:44.059689 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:44.068413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:44.094353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1357/2722] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:43 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:44.099605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:44.112270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573838-2c47-4cb4-84d7-eef74c975933 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:44.117431 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:44.118185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:44.118473 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:44.118915 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:44.119623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:44.121267 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:44.135009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1366/2723] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:44.139839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:44.155412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf05eb-0372-413f-8b48-9d031e105e2a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:44.163155 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:44.164663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:44.165148 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:44.165537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] PATH_INFO: `/v3/projects/32567c92dfb44d1484b0aee7aacf0b47/users/793865aa2107423e8fccdffb2832d9dd/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:44.197188 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32567c92dfb44d1484b0aee7aacf0b47, user_id=793865aa2107423e8fccdffb2832d9dd, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:44.202652 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:44.234432 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1358/2724] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:44 2022] PUT /identity/v3/projects/32567c92dfb44d1484b0aee7aacf0b47/users/793865aa2107423e8fccdffb2832d9dd/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:44.240413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:44.258649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c3921-b485-483d-9383-ca1b57d1123b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:44.357994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:44.359167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:44.359682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:44.360292 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:44.361704 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:44.366413 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:44.391308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1367/2725] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:44.393443 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:44.412808 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a6a84-412e-4388-96ef-a807a5d65909 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:44.421301 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53af29-0c9c-4316-a410-e8899e50e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:44.423160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53af29-0c9c-4316-a410-e8899e50e82a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:44.423870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53af29-0c9c-4316-a410-e8899e50e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:44.424237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53af29-0c9c-4316-a410-e8899e50e82a admin admin] PATH_INFO: `/v3/projects/32567c92dfb44d1484b0aee7aacf0b47/users/793865aa2107423e8fccdffb2832d9dd/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:44.428269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53af29-0c9c-4316-a410-e8899e50e82a admin admin] RBAC: Authorizing `identity:create_grant(project_id=32567c92dfb44d1484b0aee7aacf0b47, user_id=793865aa2107423e8fccdffb2832d9dd, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:44.433949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53af29-0c9c-4316-a410-e8899e50e82a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:44.457252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1359/2726] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:44 2022] PUT /identity/v3/projects/32567c92dfb44d1484b0aee7aacf0b47/users/793865aa2107423e8fccdffb2832d9dd/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:46.896030 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:46.912270 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba7630-a7fe-42ab-a217-7fcde6c22498 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:46.947329 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:47.032715 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:47.035094 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:47.035652 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:47.036199 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] PATH_INFO: `/v3/users/2834625eaa054138ba3afc3ef7f9214a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:47.041709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2834625eaa054138ba3afc3ef7f9214a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:47.044816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:47.145447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2834625eaa054138ba3afc3ef7f9214a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:47.172808 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2834625eaa054138ba3afc3ef7f9214a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:47.195232 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2834625eaa054138ba3afc3ef7f9214a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:47.197186 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2834625eaa054138ba3afc3ef7f9214a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:47.258004 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2834625eaa054138ba3afc3ef7f9214a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:47.277907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1368/2727] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:46 2022] DELETE /identity/v3/users/2834625eaa054138ba3afc3ef7f9214a => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:47.284753 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:47.285642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:47.285642 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:47.326312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:47.330312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:47.479622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:47.482092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:47.488419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1360/2728] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:47.595022 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3db39f-2f79-4b58-868a-a5085a5470c3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:47.605100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:47.606555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:47.607005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:47.607197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:47.622569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:47.625616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:47.641885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1369/2729] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:48.622180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:48.624816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:48.660742 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:48.665112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:48.691364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02df2ff-72e0-4526-8501-db205360cfe6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:48.720529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:48.727748 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:48.728620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:48.728809 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f64f0-8006-4680-82f6-40c0e7cde07a service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:48.729523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] PATH_INFO: `/v3/users/77b1c2dca3fa4d57a5258fabeeac6cdd` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:48.734638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b1c2dca3fa4d57a5258fabeeac6cdd)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:48.741786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:48.748401 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:48.750577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:48.753631 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:48.755481 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] PATH_INFO: `/v3/projects/4ccb8e176b714bb89d00adee9521890e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:48.764760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ccb8e176b714bb89d00adee9521890e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:48.767145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:48.925129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b1c2dca3fa4d57a5258fabeeac6cdd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:48.932158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:48.942368 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b1c2dca3fa4d57a5258fabeeac6cdd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:48.977397 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b1c2dca3fa4d57a5258fabeeac6cdd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:48.978803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b1c2dca3fa4d57a5258fabeeac6cdd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:48.993224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ccb8e176b714bb89d00adee9521890e', 'request_id': 'req-35c8052d-9a7c-447c-879d-f56d2c574636'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:49.002404 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac3c1fb4-d6c6-4ae5-a303-b8d054c6f385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b1c2dca3fa4d57a5258fabeeac6cdd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:49.011415 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1361/2730] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:48 2022] DELETE /identity/v3/users/77b1c2dca3fa4d57a5258fabeeac6cdd => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:49.017568 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:49.018710 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:49.019091 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ccb8e176b714bb89d00adee9521890e', 'request_id': 'req-35c8052d-9a7c-447c-879d-f56d2c574636'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:49.019265 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:49.038387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ccb8e176b714bb89d00adee9521890e', 'request_id': 'req-35c8052d-9a7c-447c-879d-f56d2c574636'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:49.055129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:49.057013 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:49.102058 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] The token cache is being invalidate because project 4ccb8e176b714bb89d00adee9521890e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:49.102557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:49.103698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-35c8052d-9a7c-447c-879d-f56d2c574636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:49.106719 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1370/2731] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:48 2022] DELETE /identity/v3/projects/4ccb8e176b714bb89d00adee9521890e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:49.115078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd25daa-838b-4583-9909-ad626bc5b66e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:49.116310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd25daa-838b-4583-9909-ad626bc5b66e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:49.116310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd25daa-838b-4583-9909-ad626bc5b66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:49.123177 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=6,keystone:SELECT=84,keystone:DELETE=10 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:49.223893 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-edd25daa-838b-4583-9909-ad626bc5b66e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:49.229428 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-edd25daa-838b-4583-9909-ad626bc5b66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:49.286638 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1362/2732] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:49.319268 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:49.342864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:49.523641 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1371/2733] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:49.525359 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad344-6e35-4942-a726-f3a73e6c2b13 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:49.526049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad344-6e35-4942-a726-f3a73e6c2b13 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:49.526432 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ad344-6e35-4942-a726-f3a73e6c2b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:49.580965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-809ad344-6e35-4942-a726-f3a73e6c2b13 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:49.583393 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-809ad344-6e35-4942-a726-f3a73e6c2b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:49.606070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1372/2734] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:49.617586 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ad344-6e35-4942-a726-f3a73e6c2b13 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:49.661407 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ad344-6e35-4942-a726-f3a73e6c2b13 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:49.692145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0196535-bc50-47d1-b459-ba110cf1c266 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:49.700604 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:49.704716 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:49.706634 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:49.708623 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:49.710549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:49.712508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:49.714000 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:49.714939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:49.719218 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:49.719975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:49.721810 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:49.764970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:49.767911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:49.771289 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1373/2735] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:49.772887 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:49.828776 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1363/2736] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:49.830033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1326eb6-32fb-4938-8aaa-76089bbc5c73 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:49.851407 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:49.852466 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:49.852870 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:49.853264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:49.854411 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:49.858456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:49.948900 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1374/2737] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:49 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:49.956340 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:50.000409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687927e9-11a0-4550-8c92-b169064e6cb2 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:50.274332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:50.275211 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:50.275764 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:50.276133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:50.278563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:50.284116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:50.368815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:50.369112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1364/2738] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:49 2022] POST /identity/v3/users => generated 287 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:50.412143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d796c19-ee70-4f19-8797-0e348e8132bd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:50.434444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:50.435677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:50.436447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:50.436925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:50.438048 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:50.440783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:50.478522 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1375/2739] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:50.483298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:50.510595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acf74d-4950-48a6-9f38-45c0cb163dea admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:50.524097 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:50.524891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:50.525180 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:50.525458 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:50.540126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:50.546161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:50.591749 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:50.595766 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:50.598641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1365/2740] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:50.599688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:50.644169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dec516-ee52-4730-82cc-afa311e990c8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:50.659826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f032c6-aa22-4c4c-b6fe-7f5f13b6f713 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:50.670691 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da2638-fba5-40af-b76f-4a66b40c73d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:50.671729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da2638-fba5-40af-b76f-4a66b40c73d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:50.672038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da2638-fba5-40af-b76f-4a66b40c73d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:50.677832 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da2638-fba5-40af-b76f-4a66b40c73d3 admin admin] PATH_INFO: `/v3/projects/a3e89b1c5f094b13b9aaf8ef19018d83/users/7dda53e6be7145299b2c42667f39a398/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:50.684204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:50.686035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:50.687886 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:50.697234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] PATH_INFO: `/v3/projects/2884447e2332431bad43869dd9af8bb2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:50.702240 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2884447e2332431bad43869dd9af8bb2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:50.705780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:50.741739 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da2638-fba5-40af-b76f-4a66b40c73d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e89b1c5f094b13b9aaf8ef19018d83, user_id=7dda53e6be7145299b2c42667f39a398, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:50.754637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da2638-fba5-40af-b76f-4a66b40c73d3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:50.787908 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1366/2741] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:50 2022] PUT /identity/v3/projects/a3e89b1c5f094b13b9aaf8ef19018d83/users/7dda53e6be7145299b2c42667f39a398/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:50.805109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:50.807372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:50.808410 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:50.878295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] MFA Rules not processed for user `7dda53e6be7145299b2c42667f39a398`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:50.880429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:50.883095 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2884447e2332431bad43869dd9af8bb2', 'request_id': 'req-888aaff6-b53e-4ac6-a73c-e1d493b51573'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:50.902219 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:50.903649 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2884447e2332431bad43869dd9af8bb2', 'request_id': 'req-888aaff6-b53e-4ac6-a73c-e1d493b51573'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:50.916745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:50.929677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2884447e2332431bad43869dd9af8bb2', 'request_id': 'req-888aaff6-b53e-4ac6-a73c-e1d493b51573'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:50.974265 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] The token cache is being invalidate because project 2884447e2332431bad43869dd9af8bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:50.976780 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:50.980504 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:50.989129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1376/2742] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:50 2022] DELETE /identity/v3/projects/2884447e2332431bad43869dd9af8bb2 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:50.994108 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:50.998594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:51.113474 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1367/2743] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:51.132363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:51.136189 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:51.253140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aaff6-b53e-4ac6-a73c-e1d493b51573 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:51.274554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:51.276361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:51.276880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:51.277392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] PATH_INFO: `/v3/users/dd2385deb64042c6aa343fbbc2aabde9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:51.281311 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2385deb64042c6aa343fbbc2aabde9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:51.285337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:51.364963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1051c01-5caa-4b70-a2e4-97462c7464d6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:51.368828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:51.369538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:51.369889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:51.370144 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:51.386744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:51.388549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:51.424896 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1368/2744] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:51.485588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2385deb64042c6aa343fbbc2aabde9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.520005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:51.522663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:51.523660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2385deb64042c6aa343fbbc2aabde9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.560688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450938e0-a21b-44b7-a443-e427a92a9abd service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:51.561528 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2385deb64042c6aa343fbbc2aabde9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.565658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2385deb64042c6aa343fbbc2aabde9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.578136 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:51.579870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:51.581102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:51.581786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] PATH_INFO: `/v3/users/7dda53e6be7145299b2c42667f39a398` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:51.584510 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dda53e6be7145299b2c42667f39a398)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:51.588048 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:51.614307 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7b6e1fbc-c34f-4e04-8068-a4f51a7672ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2385deb64042c6aa343fbbc2aabde9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.632405 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1377/2745] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:50 2022] DELETE /identity/v3/users/dd2385deb64042c6aa343fbbc2aabde9 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:51.675610 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dda53e6be7145299b2c42667f39a398'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.693121 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dda53e6be7145299b2c42667f39a398'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.709162 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dda53e6be7145299b2c42667f39a398'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.710723 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dda53e6be7145299b2c42667f39a398'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.743729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dda53e6be7145299b2c42667f39a398'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:51.756054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1369/2746] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:51 2022] DELETE /identity/v3/users/7dda53e6be7145299b2c42667f39a398 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:51.761540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427931-2875-4708-857a-35c9271cce7c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:51.762444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427931-2875-4708-857a-35c9271cce7c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:51.762994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427931-2875-4708-857a-35c9271cce7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:51.816853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0e427931-2875-4708-857a-35c9271cce7c None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:51.821964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0e427931-2875-4708-857a-35c9271cce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:52.054678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1378/2747] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:52.076072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:52.079807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:52.144154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:52.238079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5a6bd-20be-4335-8497-c82acd5cd584 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:52.250166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8610e-a22e-4727-ad6e-36a7f837a6ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:52.250166 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8610e-a22e-4727-ad6e-36a7f837a6ad service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:52.252803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8610e-a22e-4727-ad6e-36a7f837a6ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:52.253103 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8610e-a22e-4727-ad6e-36a7f837a6ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:52.262249 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427931-2875-4708-857a-35c9271cce7c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:52.264189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0e427931-2875-4708-857a-35c9271cce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:52.276197 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8610e-a22e-4727-ad6e-36a7f837a6ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:52.281237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8610e-a22e-4727-ad6e-36a7f837a6ad service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:52.295116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427931-2875-4708-857a-35c9271cce7c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:52.310935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:52.312470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:52.312973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:52.313471 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] PATH_INFO: `/v3/projects/a9442801ef474fe1a7fc266c49129efa` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:52.316626 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9442801ef474fe1a7fc266c49129efa)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:52.318160 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1370/2748] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:52.320741 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:52.325356 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:52.396548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9442801ef474fe1a7fc266c49129efa', 'request_id': 'req-1d27aeee-8201-4777-92a7-ab5909f55b3c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.411628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9442801ef474fe1a7fc266c49129efa', 'request_id': 'req-1d27aeee-8201-4777-92a7-ab5909f55b3c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.424324 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9442801ef474fe1a7fc266c49129efa', 'request_id': 'req-1d27aeee-8201-4777-92a7-ab5909f55b3c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.453202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] The token cache is being invalidate because project a9442801ef474fe1a7fc266c49129efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:52.454444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.456632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d27aeee-8201-4777-92a7-ab5909f55b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:52.460826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1379/2749] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:52 2022] DELETE /identity/v3/projects/a9442801ef474fe1a7fc266c49129efa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:52.466506 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faafd47-9cc5-46d6-a675-b97f8c5c59f4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:52.466949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faafd47-9cc5-46d6-a675-b97f8c5c59f4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:52.467363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faafd47-9cc5-46d6-a675-b97f8c5c59f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:52.522827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1faafd47-9cc5-46d6-a675-b97f8c5c59f4 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:52.531203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1faafd47-9cc5-46d6-a675-b97f8c5c59f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:52.666716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1371/2750] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:52.677514 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58fde2-6ca3-4956-a596-542bfe0f8363 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:52.678974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58fde2-6ca3-4956-a596-542bfe0f8363 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:52.679795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58fde2-6ca3-4956-a596-542bfe0f8363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:52.729164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9e58fde2-6ca3-4956-a596-542bfe0f8363 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:52.733995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9e58fde2-6ca3-4956-a596-542bfe0f8363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:52.763152 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1380/2751] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:52.769182 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faafd47-9cc5-46d6-a675-b97f8c5c59f4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:52.801558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faafd47-9cc5-46d6-a675-b97f8c5c59f4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:52.819686 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:52.820757 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:52.821243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:52.821731 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:52.827985 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:48:52.830933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:52.838784 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:52.867504 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1372/2752] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:48:52.872206 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58fde2-6ca3-4956-a596-542bfe0f8363 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:52.890293 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58fde2-6ca3-4956-a596-542bfe0f8363 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:52.896818 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:52.897608 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:52.897996 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:52.898380 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:52.899153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:52.901744 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:52.960458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1381/2753] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:52 2022] POST /identity/v3/projects => generated 369 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:52.965713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:52.996502 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd753e-711a-457c-8c80-ad5bba6bfe52 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.110106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.112200 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.112550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.113049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.113693 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.115806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.160267 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1373/2754] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:52 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:53.167196 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.185675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f11362-7201-4cfe-bffb-188771939b79 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.195211 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f0ff8-a6d2-4d13-8345-bd8d18eb90a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.196854 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f0ff8-a6d2-4d13-8345-bd8d18eb90a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.197460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f0ff8-a6d2-4d13-8345-bd8d18eb90a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.197876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f0ff8-a6d2-4d13-8345-bd8d18eb90a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.198913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f0ff8-a6d2-4d13-8345-bd8d18eb90a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.201174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f0ff8-a6d2-4d13-8345-bd8d18eb90a1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.221630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1382/2755] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:53.226099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.248069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86261b-8cd9-416c-aad3-0908cf329877 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.263116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b67a14-f511-45bc-97e0-9a64fc666f93 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.263116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b67a14-f511-45bc-97e0-9a64fc666f93 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.263472 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b67a14-f511-45bc-97e0-9a64fc666f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.272447 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.274043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.275125 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.275935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.277549 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.280990 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.302325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1374/2756] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:53.304833 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d8b67a14-f511-45bc-97e0-9a64fc666f93 None None] MFA Rules not processed for user `793865aa2107423e8fccdffb2832d9dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:53.306475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d8b67a14-f511-45bc-97e0-9a64fc666f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.306704 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.328210 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb547d6-2215-4d0a-964c-2d5ffee6211c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.341578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.343574 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.344154 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.344677 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] PATH_INFO: `/v3/projects/2d3df551b54b48429fbc1a8b0af45584/users/e2f2b2dffb0d42a7b925dfe4129b5f22/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.374613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d3df551b54b48429fbc1a8b0af45584, user_id=e2f2b2dffb0d42a7b925dfe4129b5f22, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.379584 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.406551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1375/2757] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:53 2022] PUT /identity/v3/projects/2d3df551b54b48429fbc1a8b0af45584/users/e2f2b2dffb0d42a7b925dfe4129b5f22/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:53.415879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.421294 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.437685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:53.456903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1383/2758] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:53.456903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.456903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.456903 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.506538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] MFA Rules not processed for user `e2f2b2dffb0d42a7b925dfe4129b5f22`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:53.507515 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.519164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00278a-77dc-4b13-ba4b-9a97e3e5ac5a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.527433 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.528815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.529475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.529954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] PATH_INFO: `/v3/projects/a3e89b1c5f094b13b9aaf8ef19018d83` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.533066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e89b1c5f094b13b9aaf8ef19018d83)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.536724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.551435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:53.618774 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1384/2759] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:53.622208 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e89b1c5f094b13b9aaf8ef19018d83', 'request_id': 'req-c616ef4d-8241-457b-bb53-30c16244dbdc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.627255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.629814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.635527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e89b1c5f094b13b9aaf8ef19018d83', 'request_id': 'req-c616ef4d-8241-457b-bb53-30c16244dbdc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.646051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e89b1c5f094b13b9aaf8ef19018d83', 'request_id': 'req-c616ef4d-8241-457b-bb53-30c16244dbdc'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.666485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] The token cache is being invalidate because project a3e89b1c5f094b13b9aaf8ef19018d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:53.666805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.667472 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:53.669675 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1376/2760] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:53 2022] DELETE /identity/v3/projects/a3e89b1c5f094b13b9aaf8ef19018d83 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:53.670576 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.672066 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.761141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c616ef4d-8241-457b-bb53-30c16244dbdc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.765151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f290f-dd1f-4bfc-85fb-328d04462d2b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.775815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.777300 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.777815 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.778222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.779773 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.782483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.825263 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1377/2761] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:48:53.832251 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:53.891345 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:53.904889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.905995 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.906369 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.906724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.910214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:53.959986 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ea5fa-96a3-465a-8c17-0f240b5a977b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:53.965272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:53.966273 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:53.966565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:53.966839 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:53.967558 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:53.969495 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:53.998255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1378/2762] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:48:53 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:48:54.003493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.007346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.010029 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.014264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7b8ec-f99b-4774-a072-5afa56e87336 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.020332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.021322 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.022109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.022818 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.023878 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.027798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.031575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1385/2763] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:54.047307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1379/2764] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:48:54.053989 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.064434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:54.076829 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db1171-c518-456a-aac2-99a8eabc20c4 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.084403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.085333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.085639 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.085943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] PATH_INFO: `/v3/projects/c8d7f02f31b548ef91b0d3a6480790cf/users/9c3fc4b75c4744c4944747cbd76f2027/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.119202 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d7f02f31b548ef91b0d3a6480790cf, user_id=9c3fc4b75c4744c4944747cbd76f2027, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.123511 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.144363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1386/2765] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:54 2022] PUT /identity/v3/projects/c8d7f02f31b548ef91b0d3a6480790cf/users/9c3fc4b75c4744c4944747cbd76f2027/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:54.149556 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.162329 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a370d-2767-49ca-bffd-dc487b0507e0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.249552 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896473ce-00cb-42b0-98fe-0d61a56c1532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.250864 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896473ce-00cb-42b0-98fe-0d61a56c1532 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.251310 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896473ce-00cb-42b0-98fe-0d61a56c1532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.252794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896473ce-00cb-42b0-98fe-0d61a56c1532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.254120 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896473ce-00cb-42b0-98fe-0d61a56c1532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.256462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896473ce-00cb-42b0-98fe-0d61a56c1532 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.278914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1380/2766] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:48:54.280549 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.308078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad3331-361a-4f1f-b204-45aa5bee49cc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.320353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.322817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.323963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.324079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] PATH_INFO: `/v3/projects/c8d7f02f31b548ef91b0d3a6480790cf/users/9c3fc4b75c4744c4944747cbd76f2027/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.328350 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d7f02f31b548ef91b0d3a6480790cf, user_id=9c3fc4b75c4744c4944747cbd76f2027, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.335435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.356188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1387/2767] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:48:54 2022] PUT /identity/v3/projects/c8d7f02f31b548ef91b0d3a6480790cf/users/9c3fc4b75c4744c4944747cbd76f2027/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:48:54.369928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.370424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.370798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.427289 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] MFA Rules not processed for user `9c3fc4b75c4744c4944747cbd76f2027`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:54.428541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.550946 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1381/2768] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:54.560876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.563396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.654972 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f23984-571b-4d8e-b942-c5c09e55be64 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.660423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f096d5e-763b-42f4-b45c-f116c1bf6ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.661331 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f096d5e-763b-42f4-b45c-f116c1bf6ccb service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.662062 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f096d5e-763b-42f4-b45c-f116c1bf6ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.662645 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f096d5e-763b-42f4-b45c-f116c1bf6ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.675614 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f096d5e-763b-42f4-b45c-f116c1bf6ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.677384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f096d5e-763b-42f4-b45c-f116c1bf6ccb service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.692136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1388/2769] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:48:54.763248 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:54.765003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.781929 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fafa59-f2db-4708-b07e-6fae17a06193 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:54.788484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:54.789414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:54.789749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:54.790217 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:54.793624 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:54.837064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:54.903489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:54.905001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:54.916783 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1382/2770] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:55.596997 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782345c2-c535-4378-9d61-e8877334e951 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:55.599070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782345c2-c535-4378-9d61-e8877334e951 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:55.599070 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782345c2-c535-4378-9d61-e8877334e951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:55.618880 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-782345c2-c535-4378-9d61-e8877334e951 None None] MFA Rules not processed for user `0e5ef3cd501e4baa8e6ac5f2574531fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:55.620796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-782345c2-c535-4378-9d61-e8877334e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:55.717937 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1389/2771] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:48:58.816070 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:58.901926 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d26800-84bf-4a52-b113-aab38f3b1dc7 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:58.908145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:58.909680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:58.910234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:58.910729 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] PATH_INFO: `/v3/users/e2f2b2dffb0d42a7b925dfe4129b5f22` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:58.912894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f2b2dffb0d42a7b925dfe4129b5f22)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:58.914994 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:58.974846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f2b2dffb0d42a7b925dfe4129b5f22'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:58.985455 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f2b2dffb0d42a7b925dfe4129b5f22'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:58.995151 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f2b2dffb0d42a7b925dfe4129b5f22'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:58.995917 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f2b2dffb0d42a7b925dfe4129b5f22'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.013406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f2b2dffb0d42a7b925dfe4129b5f22'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.020466 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1383/2772] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:58 2022] DELETE /identity/v3/users/e2f2b2dffb0d42a7b925dfe4129b5f22 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:59.026231 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:59.027027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:59.027027 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:59.045680 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:48:59.046934 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:59.115327 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1390/2773] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:48:59.126537 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:59.128771 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:59.188714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:59.204720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012559c8-be92-4260-90b5-ad0262a2800a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:59.210852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:59.211915 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:59.212171 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:59.212456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:59.225199 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:59.226740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:59.238279 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1384/2774] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:48:59.573389 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:59.576867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:48:59.604505 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327f941-17e1-41eb-beaf-77a6101d5e50 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:59.610260 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:59.611476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:59.611830 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:59.612183 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] PATH_INFO: `/v3/projects/2d3df551b54b48429fbc1a8b0af45584` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:59.613982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d3df551b54b48429fbc1a8b0af45584)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:59.616941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:59.632264 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:48:59.688569 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d3df551b54b48429fbc1a8b0af45584', 'request_id': 'req-1d88fe4e-aae9-4484-bf01-542beef5546a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.706275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d3df551b54b48429fbc1a8b0af45584', 'request_id': 'req-1d88fe4e-aae9-4484-bf01-542beef5546a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.713325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d3df551b54b48429fbc1a8b0af45584', 'request_id': 'req-1d88fe4e-aae9-4484-bf01-542beef5546a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.729478 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] The token cache is being invalidate because project 2d3df551b54b48429fbc1a8b0af45584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:48:59.729772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.730451 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.732071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1391/2775] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:48:59 2022] DELETE /identity/v3/projects/2d3df551b54b48429fbc1a8b0af45584 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:48:59.737931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:48:59.751077 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b341ec-18de-436d-991a-51039ec7b2fc service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:48:59.828507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:48:59.829385 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:48:59.829670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:48:59.830538 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] PATH_INFO: `/v3/users/9c3fc4b75c4744c4944747cbd76f2027` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:48:59.831787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3fc4b75c4744c4944747cbd76f2027)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:48:59.833713 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:48:59.898409 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3fc4b75c4744c4944747cbd76f2027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.914519 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3fc4b75c4744c4944747cbd76f2027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.927952 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3fc4b75c4744c4944747cbd76f2027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.928722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3fc4b75c4744c4944747cbd76f2027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.943343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c5cd6f43-c1b0-4c82-b5cd-0ee014016f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3fc4b75c4744c4944747cbd76f2027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:48:59.956038 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1385/2776] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:48:59 2022] DELETE /identity/v3/users/9c3fc4b75c4744c4944747cbd76f2027 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:00.308533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:00.311325 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:00.330963 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88fe4e-aae9-4484-bf01-542beef5546a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:00.431534 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:00.432433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.432720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.432998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] PATH_INFO: `/v3/projects/c8d7f02f31b548ef91b0d3a6480790cf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.434310 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d7f02f31b548ef91b0d3a6480790cf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:00.436246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:00.501088 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d7f02f31b548ef91b0d3a6480790cf', 'request_id': 'req-f22fa251-4768-4fcd-82f6-117de0fb70de'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.510605 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d7f02f31b548ef91b0d3a6480790cf', 'request_id': 'req-f22fa251-4768-4fcd-82f6-117de0fb70de'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.518954 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d7f02f31b548ef91b0d3a6480790cf', 'request_id': 'req-f22fa251-4768-4fcd-82f6-117de0fb70de'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.539018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] The token cache is being invalidate because project c8d7f02f31b548ef91b0d3a6480790cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:00.539323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.539970 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f22fa251-4768-4fcd-82f6-117de0fb70de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:00.541584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1392/2777] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:00 2022] DELETE /identity/v3/projects/c8d7f02f31b548ef91b0d3a6480790cf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:00.547358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b56663-ba75-4b5a-9a24-9b983b7fdad2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.547696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b56663-ba75-4b5a-9a24-9b983b7fdad2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.548169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b56663-ba75-4b5a-9a24-9b983b7fdad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.572879 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-16b56663-ba75-4b5a-9a24-9b983b7fdad2 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:00.574749 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-16b56663-ba75-4b5a-9a24-9b983b7fdad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:00.677939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1386/2778] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:00.686086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34fbb5-1235-456a-bae4-3742091a5738 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.686378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34fbb5-1235-456a-bae4-3742091a5738 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.686669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34fbb5-1235-456a-bae4-3742091a5738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.708153 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ee34fbb5-1235-456a-bae4-3742091a5738 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:00.709409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ee34fbb5-1235-456a-bae4-3742091a5738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:00.717819 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1393/2779] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:00.723435 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b56663-ba75-4b5a-9a24-9b983b7fdad2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:00.738029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b56663-ba75-4b5a-9a24-9b983b7fdad2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:00.745954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:00.747098 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.747616 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.747996 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.749109 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:00.749536 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:00.752072 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:00.754909 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:00.776334 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1387/2780] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:49:00.780667 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34fbb5-1235-456a-bae4-3742091a5738 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:00.796193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34fbb5-1235-456a-bae4-3742091a5738 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:00.801671 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:00.802384 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.802693 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.802968 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.803669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:00.805686 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:00.840033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:00.857692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1394/2781] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:00 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:00.858848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:00.872347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e77078-c940-400f-820c-c72fa1ea3f65 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:00.954387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:00.955600 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:00.955920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:00.956198 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:00.956898 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:00.959084 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:00.988822 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1388/2782] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:00 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:00.995049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.006848 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f4a97-9ada-45ae-bb3e-2de3cdef510c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.011952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.012839 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.013066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.013204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.013966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.015600 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.028190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1395/2783] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:01.032810 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.048228 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897617b-1cc1-439a-9d23-f7772d4d61e6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.053717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd51876-d6b4-4dc7-a76b-3989a381bb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.054426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd51876-d6b4-4dc7-a76b-3989a381bb33 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.054709 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd51876-d6b4-4dc7-a76b-3989a381bb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.054981 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd51876-d6b4-4dc7-a76b-3989a381bb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.055708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd51876-d6b4-4dc7-a76b-3989a381bb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.057314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd51876-d6b4-4dc7-a76b-3989a381bb33 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.069286 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1389/2784] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:01.075424 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.086858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe1eb5-6ed1-45f8-8f48-d0edfd784fb6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.092266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.093168 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.093455 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.093732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] PATH_INFO: `/v3/projects/cc647cd41e9049fba3f6d0525d5f1cb2/users/79e7947cb3c947af8644e78af9e71f8b/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.114146 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc647cd41e9049fba3f6d0525d5f1cb2, user_id=79e7947cb3c947af8644e78af9e71f8b, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.117040 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.132115 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1396/2785] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:01 2022] PUT /identity/v3/projects/cc647cd41e9049fba3f6d0525d5f1cb2/users/79e7947cb3c947af8644e78af9e71f8b/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:01.145043 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c244-e3ec-4387-9f31-25cc88e49389 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.145477 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c244-e3ec-4387-9f31-25cc88e49389 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.145891 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c244-e3ec-4387-9f31-25cc88e49389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.181237 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bf24c244-e3ec-4387-9f31-25cc88e49389 None None] MFA Rules not processed for user `79e7947cb3c947af8644e78af9e71f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:01.182534 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bf24c244-e3ec-4387-9f31-25cc88e49389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:01.276685 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1390/2786] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:01.281173 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.358826 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f00f5-fe8c-4fc2-9c2a-398cadd99c85 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.365885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.367326 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.367945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.368457 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.370104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.373445 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.405664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1397/2787] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:01.409700 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24c244-e3ec-4387-9f31-25cc88e49389 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.423272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24c244-e3ec-4387-9f31-25cc88e49389 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.498212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.499152 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.499498 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.499861 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.500958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.504666 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.513393 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:01.532763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1391/2788] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:01.537442 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.552644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de623a66-6d02-410d-bbe0-d5a411f86cd7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.558066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.558794 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.559078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.559358 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.560104 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.561743 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.575227 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1398/2789] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:01.580416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.594028 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559eace-b67b-4525-936b-5beb01d978f2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.599143 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.600060 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.600343 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.600625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] PATH_INFO: `/v3/projects/42b74537ab5446cf9e72eedb4a7a2774/users/90a8e2c16374478c9ac4a3e05a6c2093/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.618360 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b74537ab5446cf9e72eedb4a7a2774, user_id=90a8e2c16374478c9ac4a3e05a6c2093, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.621074 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.643609 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1392/2790] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:01 2022] PUT /identity/v3/projects/42b74537ab5446cf9e72eedb4a7a2774/users/90a8e2c16374478c9ac4a3e05a6c2093/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:01.648850 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.662462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21c1d3-1d05-4497-a512-68ef3eab3e80 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.736815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfa7d0-c8f3-418f-8ddb-e2ccfd113e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.737682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfa7d0-c8f3-418f-8ddb-e2ccfd113e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.738032 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfa7d0-c8f3-418f-8ddb-e2ccfd113e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.738378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfa7d0-c8f3-418f-8ddb-e2ccfd113e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.739111 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfa7d0-c8f3-418f-8ddb-e2ccfd113e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.741114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfa7d0-c8f3-418f-8ddb-e2ccfd113e9d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.760901 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1399/2791] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:01.762376 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.774087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013611e-158e-4e62-b909-72907695c20f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:01.779629 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:01.780541 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.780819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.781108 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] PATH_INFO: `/v3/projects/42b74537ab5446cf9e72eedb4a7a2774/users/90a8e2c16374478c9ac4a3e05a6c2093/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.783308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b74537ab5446cf9e72eedb4a7a2774, user_id=90a8e2c16374478c9ac4a3e05a6c2093, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:01.786052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:01.801961 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1393/2792] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:01 2022] PUT /identity/v3/projects/42b74537ab5446cf9e72eedb4a7a2774/users/90a8e2c16374478c9ac4a3e05a6c2093/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:01.813764 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff4764-a0a8-45be-8f16-ce4b398e729b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:01.814858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff4764-a0a8-45be-8f16-ce4b398e729b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:01.815155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff4764-a0a8-45be-8f16-ce4b398e729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:01.850673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-35ff4764-a0a8-45be-8f16-ce4b398e729b None None] MFA Rules not processed for user `90a8e2c16374478c9ac4a3e05a6c2093`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:01.852191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-35ff4764-a0a8-45be-8f16-ce4b398e729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:01.880333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:01.954923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1400/2793] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:01.966083 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:01.967962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:02.046925 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ae2d4-eb92-4571-909f-1897de883a3c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:02.052005 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae09f58-2c5e-4137-9d23-99f3845cf04f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:02.052681 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae09f58-2c5e-4137-9d23-99f3845cf04f service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.052975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae09f58-2c5e-4137-9d23-99f3845cf04f service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.053253 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae09f58-2c5e-4137-9d23-99f3845cf04f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.063987 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae09f58-2c5e-4137-9d23-99f3845cf04f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:02.065483 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae09f58-2c5e-4137-9d23-99f3845cf04f service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:02.077264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1394/2794] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:02.122166 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff4764-a0a8-45be-8f16-ce4b398e729b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:02.208221 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff4764-a0a8-45be-8f16-ce4b398e729b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:02.214931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:02.216421 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.216923 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.217353 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] PATH_INFO: `/v3/users/79e7947cb3c947af8644e78af9e71f8b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.219130 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e7947cb3c947af8644e78af9e71f8b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:02.222800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:02.290725 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e7947cb3c947af8644e78af9e71f8b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.300172 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e7947cb3c947af8644e78af9e71f8b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.315867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e7947cb3c947af8644e78af9e71f8b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.316779 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e7947cb3c947af8644e78af9e71f8b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.332748 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e7947cb3c947af8644e78af9e71f8b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:02.340402 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1401/2795] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:02 2022] DELETE /identity/v3/users/79e7947cb3c947af8644e78af9e71f8b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:02.346740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.347126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.347535 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.368617 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:02.370387 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:02.449530 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1395/2796] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:02.460361 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:02.461965 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:02.528254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:02.550127 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d391dd-0b48-43af-913c-2918416d359b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:02.555399 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:02.556128 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.556510 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.556890 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.570597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:02.573160 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:02.585018 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1402/2797] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:02.955325 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:02.957718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:02.977446 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f31ca3-532b-447c-b5a4-67c655129621 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:02.983679 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:02.984559 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:02.984842 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:02.985122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] PATH_INFO: `/v3/projects/cc647cd41e9049fba3f6d0525d5f1cb2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:02.986416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc647cd41e9049fba3f6d0525d5f1cb2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:02.988379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:03.043720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc647cd41e9049fba3f6d0525d5f1cb2', 'request_id': 'req-0761123c-bfe4-43a1-a09e-5cb2f7944071'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.047328 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:03.055512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc647cd41e9049fba3f6d0525d5f1cb2', 'request_id': 'req-0761123c-bfe4-43a1-a09e-5cb2f7944071'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.069595 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc647cd41e9049fba3f6d0525d5f1cb2', 'request_id': 'req-0761123c-bfe4-43a1-a09e-5cb2f7944071'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.096379 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] The token cache is being invalidate because project cc647cd41e9049fba3f6d0525d5f1cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:03.096816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.097705 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.100075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1396/2798] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:02 2022] DELETE /identity/v3/projects/cc647cd41e9049fba3f6d0525d5f1cb2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:03.107770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:03.127161 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c95b4-9c08-456b-b674-b05789e65be6 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:03.234568 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:03.235673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:03.236531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:03.236891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] PATH_INFO: `/v3/users/90a8e2c16374478c9ac4a3e05a6c2093` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:03.238863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a8e2c16374478c9ac4a3e05a6c2093)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:03.241158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:03.326092 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a8e2c16374478c9ac4a3e05a6c2093'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.334284 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a8e2c16374478c9ac4a3e05a6c2093'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.343757 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a8e2c16374478c9ac4a3e05a6c2093'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.344663 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a8e2c16374478c9ac4a3e05a6c2093'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.364330 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-de84dbdb-a05c-4a91-862e-117c8c279415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a8e2c16374478c9ac4a3e05a6c2093'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:03.375644 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1403/2799] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:03 2022] DELETE /identity/v3/users/90a8e2c16374478c9ac4a3e05a6c2093 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:03.981295 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:03.983470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:04.000935 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761123c-bfe4-43a1-a09e-5cb2f7944071 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.073793 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.074805 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.075181 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.075698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] PATH_INFO: `/v3/projects/42b74537ab5446cf9e72eedb4a7a2774` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.077201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b74537ab5446cf9e72eedb4a7a2774)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.079358 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.144565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b74537ab5446cf9e72eedb4a7a2774', 'request_id': 'req-1dd4eb54-6180-4310-9dfc-e8516e1684d3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:04.155054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b74537ab5446cf9e72eedb4a7a2774', 'request_id': 'req-1dd4eb54-6180-4310-9dfc-e8516e1684d3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:04.168247 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b74537ab5446cf9e72eedb4a7a2774', 'request_id': 'req-1dd4eb54-6180-4310-9dfc-e8516e1684d3'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:04.187660 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] The token cache is being invalidate because project 42b74537ab5446cf9e72eedb4a7a2774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:04.187968 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:04.188944 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1dd4eb54-6180-4310-9dfc-e8516e1684d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:04.192004 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1397/2800] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:03 2022] DELETE /identity/v3/projects/42b74537ab5446cf9e72eedb4a7a2774 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:04.199006 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3812e58-ec76-4bd4-a92a-199bd5354f81 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.199823 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3812e58-ec76-4bd4-a92a-199bd5354f81 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.200025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3812e58-ec76-4bd4-a92a-199bd5354f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.220703 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d3812e58-ec76-4bd4-a92a-199bd5354f81 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:04.222480 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d3812e58-ec76-4bd4-a92a-199bd5354f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:04.302935 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:04.314679 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1404/2801] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:04.323112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18782f-7129-4399-a472-4dbb0146658a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.323402 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18782f-7129-4399-a472-4dbb0146658a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.323692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18782f-7129-4399-a472-4dbb0146658a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.349204 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5f18782f-7129-4399-a472-4dbb0146658a None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:04.350490 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5f18782f-7129-4399-a472-4dbb0146658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:04.364570 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1398/2802] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:04.369798 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3812e58-ec76-4bd4-a92a-199bd5354f81 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.380613 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3812e58-ec76-4bd4-a92a-199bd5354f81 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.385792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.386705 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.386966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.387454 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.388599 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:04.389084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.390885 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.410041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1405/2803] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 17 11:49:04.411766 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18782f-7129-4399-a472-4dbb0146658a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.422836 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18782f-7129-4399-a472-4dbb0146658a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.428021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.428712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.428991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.429272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.430138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.432130 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.468635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1399/2804] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:04.474994 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.486772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffd06a-6067-4d80-b9e1-1c084cb277fb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.566535 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.567317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.567655 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.567982 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.568840 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.571019 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.597250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1406/2805] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:04.605754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.618082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0a041-bc9e-4959-abd7-af1104c5b460 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.623505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.624353 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.624658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.624949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.625820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.628774 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.647369 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1400/2806] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:04.653452 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.666828 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb0bc-de05-4904-a39c-7aea5dc2b36c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.672403 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0a2d7-6c23-43d2-b606-61abb969980d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.673273 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a2d7-6c23-43d2-b606-61abb969980d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.674045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a2d7-6c23-43d2-b606-61abb969980d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.674308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0a2d7-6c23-43d2-b606-61abb969980d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.675086 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0a2d7-6c23-43d2-b606-61abb969980d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.676769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0a2d7-6c23-43d2-b606-61abb969980d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.689677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1407/2807] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:04.695187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.709939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5a8b-ca5a-4b9f-ab35-923c0c533bd4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.715740 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.716754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.717030 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.717371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] PATH_INFO: `/v3/projects/c7492a1c88cc43e294639b368954fd80/users/8a9d97e2ab8b4e19a1289aee4738c5ff/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.738945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7492a1c88cc43e294639b368954fd80, user_id=8a9d97e2ab8b4e19a1289aee4738c5ff, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:04.742092 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:04.762342 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1401/2808] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:04 2022] PUT /identity/v3/projects/c7492a1c88cc43e294639b368954fd80/users/8a9d97e2ab8b4e19a1289aee4738c5ff/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:04.783897 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4b5ed-b7e4-435f-b42a-db9a0f7985ff None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.784255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4b5ed-b7e4-435f-b42a-db9a0f7985ff None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.784541 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4b5ed-b7e4-435f-b42a-db9a0f7985ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:04.816368 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-86d4b5ed-b7e4-435f-b42a-db9a0f7985ff None None] MFA Rules not processed for user `8a9d97e2ab8b4e19a1289aee4738c5ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:04.817635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-86d4b5ed-b7e4-435f-b42a-db9a0f7985ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:04.898907 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1408/2809] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:04.910507 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:04.912339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:04.927525 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:04.985557 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fc986-b3a0-4ed4-9f3d-e6132e365cf7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:04.990496 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4ab97-4ce2-4f20-8ade-17d33c66d89e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:04.991290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ab97-4ce2-4f20-8ade-17d33c66d89e service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:04.991768 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ab97-4ce2-4f20-8ade-17d33c66d89e service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:04.992095 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4ab97-4ce2-4f20-8ade-17d33c66d89e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:05.002308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4ab97-4ce2-4f20-8ade-17d33c66d89e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:05.003806 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4ab97-4ce2-4f20-8ade-17d33c66d89e service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:05.013551 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1402/2810] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:13.811191 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4b5ed-b7e4-435f-b42a-db9a0f7985ff None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:13.879475 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:13.886556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4b5ed-b7e4-435f-b42a-db9a0f7985ff None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:13.892507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:13.893437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:13.893724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:13.894039 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] PATH_INFO: `/v3/users/8a9d97e2ab8b4e19a1289aee4738c5ff` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:13.895345 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9d97e2ab8b4e19a1289aee4738c5ff)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:13.897497 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:13.957222 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9d97e2ab8b4e19a1289aee4738c5ff'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:13.967815 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9d97e2ab8b4e19a1289aee4738c5ff'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:13.979266 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9d97e2ab8b4e19a1289aee4738c5ff'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:13.979936 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9d97e2ab8b4e19a1289aee4738c5ff'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.002801 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9d97e2ab8b4e19a1289aee4738c5ff'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.011787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1409/2811] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:49:13 2022] DELETE /identity/v3/users/8a9d97e2ab8b4e19a1289aee4738c5ff => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:14.016632 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.017956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.017956 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.041424 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:14.042708 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.112102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1403/2812] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:14.122913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:14.124387 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.188664 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd815996-bcf0-4b39-b3a3-c49af4673e32 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:14.192235 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:14.192889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.193174 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.193468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.202857 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:14.204295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:14.215557 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1410/2813] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:14.503645 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:14.505366 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.554684 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:14.556899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.578494 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ec0d6-23a0-4ee7-987a-19de7ca24313 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:14.587418 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:14.588456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.588787 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.589071 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] PATH_INFO: `/v3/projects/c7492a1c88cc43e294639b368954fd80` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.590346 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7492a1c88cc43e294639b368954fd80)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:14.592341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:14.606601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a25e4-4465-421a-aee1-3bc0ad06d214 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:14.611359 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75ef04-33bd-40aa-a1eb-aae1a98ef4ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:14.612102 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75ef04-33bd-40aa-a1eb-aae1a98ef4ac service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.612400 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75ef04-33bd-40aa-a1eb-aae1a98ef4ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.612665 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75ef04-33bd-40aa-a1eb-aae1a98ef4ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.617039 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9c75ef04-33bd-40aa-a1eb-aae1a98ef4ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.663282 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7492a1c88cc43e294639b368954fd80', 'request_id': 'req-c8c724b3-3617-444c-849b-01eb47082554'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.674052 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7492a1c88cc43e294639b368954fd80', 'request_id': 'req-c8c724b3-3617-444c-849b-01eb47082554'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.680279 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7492a1c88cc43e294639b368954fd80', 'request_id': 'req-c8c724b3-3617-444c-849b-01eb47082554'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.688586 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:14.694950 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] The token cache is being invalidate because project c7492a1c88cc43e294639b368954fd80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:49:14.695377 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.696419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c8c724b3-3617-444c-849b-01eb47082554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:49:14.699862 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1411/2814] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:49:14 2022] DELETE /identity/v3/projects/c7492a1c88cc43e294639b368954fd80 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:49:14.707660 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d66cb6-4f8f-4e80-a818-9ca33e90352b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.708320 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d66cb6-4f8f-4e80-a818-9ca33e90352b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.709042 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d66cb6-4f8f-4e80-a818-9ca33e90352b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.716021 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:14.733055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-54d66cb6-4f8f-4e80-a818-9ca33e90352b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:14.734363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-54d66cb6-4f8f-4e80-a818-9ca33e90352b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.736869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75ef04-33bd-40aa-a1eb-aae1a98ef4ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:14.738389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75ef04-33bd-40aa-a1eb-aae1a98ef4ac service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:14.762372 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1404/2815] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:14.830214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1412/2816] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:14.840026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341adfc-4621-463b-aa37-6157b3815653 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.840473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341adfc-4621-463b-aa37-6157b3815653 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.840983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341adfc-4621-463b-aa37-6157b3815653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.866755 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2341adfc-4621-463b-aa37-6157b3815653 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:14.868052 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2341adfc-4621-463b-aa37-6157b3815653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:14.877087 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1405/2817] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:14.881979 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d66cb6-4f8f-4e80-a818-9ca33e90352b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:14.901057 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d66cb6-4f8f-4e80-a818-9ca33e90352b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:14.909409 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:14.910607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.911125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.911632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.913051 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:49:14.913616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:14.916763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:14.938075 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1413/2818] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:49:14.943090 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341adfc-4621-463b-aa37-6157b3815653 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:14.953626 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341adfc-4621-463b-aa37-6157b3815653 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:14.959091 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:14.959828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:14.960116 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:14.960394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:14.961179 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:14.965904 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:14.993938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1406/2819] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:49:14 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:49:14.999073 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.012845 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4f70a-98de-4e96-a91d-2a3d26fa3db5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.082820 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.083502 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.083789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.084066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.084771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:15.086732 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:15.120492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1414/2820] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:49:14 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:49:15.125406 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.142101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67566ad-95d1-47a1-af5a-ed3eed0dcf57 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.147852 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.148800 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.149105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.149459 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.150201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:15.152284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:15.170873 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.173949 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1407/2821] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:49:15.174301 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:15.177914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.191482 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeff72-c441-4165-a213-0ad821b1b80a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.197694 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.198480 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.198765 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.199191 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.199945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:15.201565 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:15.219830 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1408/2822] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:49:15.222053 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.233026 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e441e-5269-401d-aee5-1c44fd1dfe7e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.238696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a4e12-5992-4764-bf15-fd7fcebd0e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.239594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a4e12-5992-4764-bf15-fd7fcebd0e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.239873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a4e12-5992-4764-bf15-fd7fcebd0e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.240149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a4e12-5992-4764-bf15-fd7fcebd0e10 admin admin] PATH_INFO: `/v3/projects/11be2d07968147078ebbad72b1b8066f/users/404ca8f64c63474480966c07e7052ae2/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.258628 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691a30a-aae4-4d25-88ab-1fd7a4303a5b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.259318 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a4e12-5992-4764-bf15-fd7fcebd0e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11be2d07968147078ebbad72b1b8066f, user_id=404ca8f64c63474480966c07e7052ae2, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:15.262633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a4e12-5992-4764-bf15-fd7fcebd0e10 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:15.264049 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.265444 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.265941 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.266984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.270855 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:15.279527 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1409/2823] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:49:15 2022] PUT /identity/v3/projects/11be2d07968147078ebbad72b1b8066f/users/404ca8f64c63474480966c07e7052ae2/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:49:15.286002 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.286374 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.286653 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.286795 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:15.288831 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:15.301398 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1415/2824] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:15.310505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:15.311962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:15.379817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1410/2825] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:15.392074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.394360 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:15.408243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28ad28-b1e5-4ed5-a337-10ef2aa8cb42 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.432730 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:15.483364 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.484335 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.484745 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.485131 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.501796 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:15.504555 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:15.522712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1416/2826] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:15.921532 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.924450 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:15.931647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:15.950579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3d843-d5c7-4e0b-a738-60a419c474ab service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:15.956669 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:15.957805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:15.958708 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:15.960512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:15.966224 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:16.079111 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca92af-9752-42ed-b4ee-655590b3a034 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.086951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ead71-10b6-490b-8f78-6f88ac4872f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.087722 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ead71-10b6-490b-8f78-6f88ac4872f5 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.088123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ead71-10b6-490b-8f78-6f88ac4872f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.088505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ead71-10b6-490b-8f78-6f88ac4872f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.091539 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-368ead71-10b6-490b-8f78-6f88ac4872f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:16.112800 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.113232 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ead71-10b6-490b-8f78-6f88ac4872f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.116281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ead71-10b6-490b-8f78-6f88ac4872f5 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.116682 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.136099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1411/2827] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:16.138078 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1417/2828] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:16.351302 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0762-d915-41b4-980b-0246182676ad None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.351697 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0762-d915-41b4-980b-0246182676ad None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.352169 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0762-d915-41b4-980b-0246182676ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.373240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:16.396112 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-902b0762-d915-41b4-980b-0246182676ad None None] MFA Rules not processed for user `404ca8f64c63474480966c07e7052ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:16.397449 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-902b0762-d915-41b4-980b-0246182676ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:16.519029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1412/2829] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:16.532234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:16.534675 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:16.618129 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a96ccc-7bb4-4979-9a1b-67e39ffb7af2 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:16.622554 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:16.623337 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:16.623761 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:16.624055 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:16.637433 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:16.639834 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:16.653677 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1418/2830] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:18.131291 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.131963 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.132206 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.191126 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] MFA Rules not processed for user `404ca8f64c63474480966c07e7052ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:18.193419 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:18.207780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1413/2831] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:49:18.246690 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:18.260842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e138e4-fd12-422f-abed-50fca7bc52b7 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:18.265113 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:18.266054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:18.266524 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:18.266962 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:18.269817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:18.288175 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:18.291915 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:18.309437 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1419/2832] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:49:22.886134 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:22.888401 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:22.907516 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f80f0-a2fc-4b07-ac06-a123acf3ba6d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:22.913082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:22.913743 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:22.914054 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:22.914330 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:22.916319 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:22.932332 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:22.935651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:22.952943 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1414/2833] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:26.187397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:26.188942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:26.224154 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c7457-5498-4ab0-b1de-9045829d9255 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:26.235538 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1c929-1793-4e6e-9268-4e9f65c03478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:26.243783 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c929-1793-4e6e-9268-4e9f65c03478 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:26.244296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c929-1793-4e6e-9268-4e9f65c03478 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:26.244931 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c929-1793-4e6e-9268-4e9f65c03478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:26.248562 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-54d1c929-1793-4e6e-9268-4e9f65c03478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:26.273192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1c929-1793-4e6e-9268-4e9f65c03478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:26.276592 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1c929-1793-4e6e-9268-4e9f65c03478 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:26.297752 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1420/2834] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:28.791695 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:28.806227 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849273d-9c73-4805-947b-5f7ad82f2071 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:28.810207 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11fd46-fdeb-43a7-aa60-a9753506a0a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:28.810799 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11fd46-fdeb-43a7-aa60-a9753506a0a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:28.811085 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11fd46-fdeb-43a7-aa60-a9753506a0a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:28.811362 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11fd46-fdeb-43a7-aa60-a9753506a0a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:28.813298 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7c11fd46-fdeb-43a7-aa60-a9753506a0a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:28.898233 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11fd46-fdeb-43a7-aa60-a9753506a0a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:28.900887 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11fd46-fdeb-43a7-aa60-a9753506a0a8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:28.924339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1415/2835] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:49:28 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:49:34.723905 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:34.724495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:34.724495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:34.742423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] MFA Rules not processed for user `404ca8f64c63474480966c07e7052ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:34.743654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:34.751576 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1421/2836] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:49:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:38.916407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=68 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:44.737397 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:49:56.274133 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83beb923-652d-42b7-88c1-4231d013d516 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.274761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83beb923-652d-42b7-88c1-4231d013d516 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.274761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83beb923-652d-42b7-88c1-4231d013d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.294718 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-83beb923-652d-42b7-88c1-4231d013d516 None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:49:56.295957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-83beb923-652d-42b7-88c1-4231d013d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:56.369445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1416/2837] 10.222.0.25 () {58 vars in 1009 bytes} [Fri Jun 17 11:49:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:49:56.426164 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:49:56.427720 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:49:56.517520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88c161-ce67-4b3a-8e49-9d67cff01c36 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:49:56.523406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:49:56.524114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:49:56.524440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:49:56.524759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:49:56.534612 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:49:56.536894 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:49:56.546190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1422/2838] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:49:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 17 11:50:06.361187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=25 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:06.542617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=32 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:13.197123 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98935620-074d-4397-8197-23112fcba102 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:13.197651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98935620-074d-4397-8197-23112fcba102 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:13.197761 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98935620-074d-4397-8197-23112fcba102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:13.216071 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-98935620-074d-4397-8197-23112fcba102 None None] MFA Rules not processed for user `84e3617bb236496783475fad6693d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:13.217448 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-98935620-074d-4397-8197-23112fcba102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:13.226135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1417/2839] 10.222.0.25 () {58 vars in 967 bytes} [Fri Jun 17 11:50:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:13.239295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:13.250945 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2bb69-13a0-417c-9bd9-3f57c727d609 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:13.256952 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3119ad342840ddb0258e124be55597', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:13.257633 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:13.258616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:13.258616 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:13.268465 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:13.270185 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:13.279683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1423/2840] 10.222.0.25 () {60 vars in 1384 bytes} [Fri Jun 17 11:50:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 17 11:50:15.360770 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98935620-074d-4397-8197-23112fcba102 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:15.378712 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98935620-074d-4397-8197-23112fcba102 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:15.386215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:15.387105 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:15.387394 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:15.387676 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] PATH_INFO: `/v3/users/404ca8f64c63474480966c07e7052ae2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:15.388906 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404ca8f64c63474480966c07e7052ae2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:15.390888 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:15.448529 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404ca8f64c63474480966c07e7052ae2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.458187 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404ca8f64c63474480966c07e7052ae2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.466603 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404ca8f64c63474480966c07e7052ae2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.467281 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404ca8f64c63474480966c07e7052ae2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.481833 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6d9c7279-9fdc-4f83-972b-472cdfb57624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404ca8f64c63474480966c07e7052ae2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.488853 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1418/2841] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:15 2022] DELETE /identity/v3/users/404ca8f64c63474480966c07e7052ae2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:15.704135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:15.719291 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd54fc-5f27-4d38-8240-ae2c6f34bc25 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:15.791474 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:15.792420 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:15.792789 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:15.793074 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] PATH_INFO: `/v3/projects/11be2d07968147078ebbad72b1b8066f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:15.794378 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=11be2d07968147078ebbad72b1b8066f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:15.796365 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:15.854386 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11be2d07968147078ebbad72b1b8066f', 'request_id': 'req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.865263 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11be2d07968147078ebbad72b1b8066f', 'request_id': 'req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.873491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11be2d07968147078ebbad72b1b8066f', 'request_id': 'req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.893723 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] The token cache is being invalidate because project 11be2d07968147078ebbad72b1b8066f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:15.894109 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.894928 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-070224f9-79a5-46d5-b6d0-a9e2203b3d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:15.896763 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1424/2842] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:15 2022] DELETE /identity/v3/projects/11be2d07968147078ebbad72b1b8066f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:15.902501 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7de517-c3de-4e7c-b038-361b0c217692 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:15.902983 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7de517-c3de-4e7c-b038-361b0c217692 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:15.903728 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7de517-c3de-4e7c-b038-361b0c217692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:15.925651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-be7de517-c3de-4e7c-b038-361b0c217692 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:15.926928 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-be7de517-c3de-4e7c-b038-361b0c217692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:15.994606 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1419/2843] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:16.003896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757f1f2-e24c-4e49-b376-eaf0c0e2fb8b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.004158 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757f1f2-e24c-4e49-b376-eaf0c0e2fb8b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.004441 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757f1f2-e24c-4e49-b376-eaf0c0e2fb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.030283 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6757f1f2-e24c-4e49-b376-eaf0c0e2fb8b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:16.032059 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6757f1f2-e24c-4e49-b376-eaf0c0e2fb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:16.044814 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1425/2844] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:16.052011 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7de517-c3de-4e7c-b038-361b0c217692 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.067064 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7de517-c3de-4e7c-b038-361b0c217692 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.072635 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.073378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.073663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.073969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.074714 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:50:16.075022 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.076948 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.093441 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1420/2845] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:50:16.098875 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757f1f2-e24c-4e49-b376-eaf0c0e2fb8b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.112357 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757f1f2-e24c-4e49-b376-eaf0c0e2fb8b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.119101 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.120219 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.120771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.121223 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.122543 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.127884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.157392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1426/2846] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:50:16 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:50:16.162161 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.172962 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db403506-121e-425d-98ea-620eedfa67c3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.228945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:16.240438 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.241145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.241641 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.242008 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.242803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.244816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.267991 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1421/2847] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:50:16 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:50:16.272560 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.283867 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e92a40-1323-46fd-9129-ec80fd4968cb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.292992 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.293692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.294009 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.294295 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.295035 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.296659 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.319065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1427/2848] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:50:16.323321 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.334809 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad648df-3c05-4b7c-b703-76f40a4ef956 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.341637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eabf27-0a99-49d0-ab44-947539b36a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.342933 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eabf27-0a99-49d0-ab44-947539b36a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.343440 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eabf27-0a99-49d0-ab44-947539b36a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.343894 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eabf27-0a99-49d0-ab44-947539b36a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.344640 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eabf27-0a99-49d0-ab44-947539b36a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.346827 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eabf27-0a99-49d0-ab44-947539b36a73 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.361456 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1422/2849] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:50:16.367344 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.381816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c57c8-4b31-4494-8f73-a88e8321020c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.395519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.396064 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.396589 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.397100 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] PATH_INFO: `/v3/projects/c3d7c860b43d41339a137db43e07d86f/users/a16573713a814feca994afd2f3867aea/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.419140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d7c860b43d41339a137db43e07d86f, user_id=a16573713a814feca994afd2f3867aea, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.421919 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.437065 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1428/2850] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:50:16 2022] PUT /identity/v3/projects/c3d7c860b43d41339a137db43e07d86f/users/a16573713a814feca994afd2f3867aea/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:16.450323 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9d8a-ba2d-4b33-a62e-6bc4ec73befa None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.450938 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9d8a-ba2d-4b33-a62e-6bc4ec73befa None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.451412 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9d8a-ba2d-4b33-a62e-6bc4ec73befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.488039 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8f3d9d8a-ba2d-4b33-a62e-6bc4ec73befa None None] MFA Rules not processed for user `a16573713a814feca994afd2f3867aea`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:16.489284 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8f3d9d8a-ba2d-4b33-a62e-6bc4ec73befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:16.565280 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1423/2851] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:50:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:50:16.577404 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:16.579973 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:16.594791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:16.664458 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e705ac-aea7-4748-b20d-df443fc27c66 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:16.670811 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17908559-fd07-4e23-bdc8-05b58e44a328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17154b0972cb4534a5c379babcb9d2f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:16.671975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17908559-fd07-4e23-bdc8-05b58e44a328 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:16.672468 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17908559-fd07-4e23-bdc8-05b58e44a328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:16.672974 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17908559-fd07-4e23-bdc8-05b58e44a328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:16.695771 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17908559-fd07-4e23-bdc8-05b58e44a328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:16.698114 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17908559-fd07-4e23-bdc8-05b58e44a328 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:16.708902 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1429/2852] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:50:24.146378 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d9d8a-ba2d-4b33-a62e-6bc4ec73befa None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:24.227747 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d9d8a-ba2d-4b33-a62e-6bc4ec73befa None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:24.232590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:24.233590 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:24.233895 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:24.234172 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] PATH_INFO: `/v3/users/a16573713a814feca994afd2f3867aea` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:24.235413 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16573713a814feca994afd2f3867aea)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:24.237421 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:24.291122 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16573713a814feca994afd2f3867aea'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:24.299426 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16573713a814feca994afd2f3867aea'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:24.308786 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16573713a814feca994afd2f3867aea'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:24.309550 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16573713a814feca994afd2f3867aea'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:24.310919 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:24.328115 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16573713a814feca994afd2f3867aea'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:24.335215 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1424/2853] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:50:24 2022] DELETE /identity/v3/users/a16573713a814feca994afd2f3867aea => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:50:24.340047 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:24.340577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:24.340722 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:24.360477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:50:24.361695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:24.431891 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1430/2854] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 17 11:50:24.442335 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:24.443870 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:24.522663 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf62348-9b6e-4cee-a383-7d95e2985f01 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:24.528889 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:24.529798 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:24.530234 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:24.530658 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:24.542564 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:24.544063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:24.555099 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1425/2855] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:50:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:50:24.917067 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:50:24.919151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:50:24.932340 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580a5ea-e26d-4adc-a095-79b3607f4c21 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:50:24.938712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:50:24.939698 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:50:24.940491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:50:24.940938 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] PATH_INFO: `/v3/projects/c3d7c860b43d41339a137db43e07d86f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:50:24.942079 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d7c860b43d41339a137db43e07d86f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:50:24.944033 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:50:25.014278 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d7c860b43d41339a137db43e07d86f', 'request_id': 'req-2160d70c-7850-480b-af4b-5d1835ce8d5f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:25.023462 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d7c860b43d41339a137db43e07d86f', 'request_id': 'req-2160d70c-7850-480b-af4b-5d1835ce8d5f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:25.039134 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d7c860b43d41339a137db43e07d86f', 'request_id': 'req-2160d70c-7850-480b-af4b-5d1835ce8d5f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:25.051434 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] The token cache is being invalidate because project c3d7c860b43d41339a137db43e07d86f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:50:25.051721 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:25.052396 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:50:25.054041 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1431/2856] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:50:24 2022] DELETE /identity/v3/projects/c3d7c860b43d41339a137db43e07d86f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:50:34.550069 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:50:35.047769 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:51:51.760049 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:51.822924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c0959-7888-4c60-91b7-eda9598a4a28 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:51.912607 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:51.913274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:51.913562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:51.913937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:51.915849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:51.963754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:51.965201 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:51.975137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1426/2857] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:51:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:51:52.056323 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:51:52.057806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:52.079017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160d70c-7850-480b-af4b-5d1835ce8d5f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:51:52.086001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:51:52.087120 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:51:52.087621 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:51:52.088116 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:51:52.091238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:51:52.229863 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:51:52.231229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:51:52.243422 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1432/2858] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:51:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:52:01.970249 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=47 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:02.237776 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=48 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:52:48.146951 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:52:48.148464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:48.252264 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd36db-1efb-40f0-86f8-763eadd0a804 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:52:48.257659 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:52:48.258352 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:52:48.258633 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:52:48.258911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:52:48.260819 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:52:48.273486 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:52:48.275203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:52:48.287347 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1427/2859] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:52:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 17 11:52:58.281586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=41 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:26.804906 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:26.806683 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:26.855045 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf80a-92d8-4516-af3a-d39d7666d3da service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:26.860588 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:26.861246 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:26.861533 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:26.861816 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:26.863692 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:26.877590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:26.879435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:26.891197 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1433/2860] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:32.268680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:32.280716 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a634c-b10c-402d-a891-5ba2170b54c3 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:32.287290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:32.287824 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:32.288375 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:32.288970 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:32.291050 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:32.303817 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:32.305434 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:32.338082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1428/2861] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:35.522734 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:35.525381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:35.622214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3476-9881-4789-a09b-d04babc06e5c service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:35.626842 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:35.627772 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:35.628038 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:35.628376 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] PATH_INFO: `/v3/users/0e5ef3cd501e4baa8e6ac5f2574531fb` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:35.629517 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e5ef3cd501e4baa8e6ac5f2574531fb)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:35.631537 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:35.681255 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e5ef3cd501e4baa8e6ac5f2574531fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:35.688204 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e5ef3cd501e4baa8e6ac5f2574531fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:35.695492 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e5ef3cd501e4baa8e6ac5f2574531fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:35.696046 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e5ef3cd501e4baa8e6ac5f2574531fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:35.708015 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e5ef3cd501e4baa8e6ac5f2574531fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:35.713916 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1434/2862] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:53:35 2022] DELETE /identity/v3/users/0e5ef3cd501e4baa8e6ac5f2574531fb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 17 11:53:35.884957 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:35.900571 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6857-1767-43cd-91da-f6d0a0b8a28a service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:35.960484 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:35.961274 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:35.961562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:35.961865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] PATH_INFO: `/v3/projects/f8ba2c3616c24a408c4003277db19262` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:35.963149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ba2c3616c24a408c4003277db19262)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:35.964958 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:36.010920 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ba2c3616c24a408c4003277db19262', 'request_id': 'req-fe92449f-6bc0-471b-8459-be8da2393c53'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.018569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ba2c3616c24a408c4003277db19262', 'request_id': 'req-fe92449f-6bc0-471b-8459-be8da2393c53'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.025627 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ba2c3616c24a408c4003277db19262', 'request_id': 'req-fe92449f-6bc0-471b-8459-be8da2393c53'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.038270 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] The token cache is being invalidate because project f8ba2c3616c24a408c4003277db19262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:53:36.038566 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.039222 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.040778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1429/2863] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:53:35 2022] DELETE /identity/v3/projects/f8ba2c3616c24a408c4003277db19262 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:53:36.044806 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:36.046285 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:36.059539 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d75fba-774c-419d-874f-ac8358058b42 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:36.071563 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:36.117739 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:36.118647 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:36.118933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:36.119212 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] PATH_INFO: `/v3/users/793865aa2107423e8fccdffb2832d9dd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:36.120419 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=793865aa2107423e8fccdffb2832d9dd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:36.122470 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:36.170008 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793865aa2107423e8fccdffb2832d9dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.177192 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793865aa2107423e8fccdffb2832d9dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.184597 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793865aa2107423e8fccdffb2832d9dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.185309 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793865aa2107423e8fccdffb2832d9dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.197578 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793865aa2107423e8fccdffb2832d9dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.203520 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1435/2864] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 11:53:36 2022] DELETE /identity/v3/users/793865aa2107423e8fccdffb2832d9dd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:53:36.254384 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:36.256333 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:36.313229 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92449f-6bc0-471b-8459-be8da2393c53 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:36.342978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:36.362706 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfef79-40b6-48cd-8de7-1a92ffd60161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:36.363363 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfef79-40b6-48cd-8de7-1a92ffd60161 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:36.363651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfef79-40b6-48cd-8de7-1a92ffd60161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:36.363931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfef79-40b6-48cd-8de7-1a92ffd60161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:36.365744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0bcfef79-40b6-48cd-8de7-1a92ffd60161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:36.431569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfef79-40b6-48cd-8de7-1a92ffd60161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:36.433192 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcfef79-40b6-48cd-8de7-1a92ffd60161 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:36.445057 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1430/2865] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:36.583106 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:36.584635 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:36.599287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689bd59-2685-4ac4-be84-9a7549f0a320 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:36.605817 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:36.607142 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:36.607491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:36.607759 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] PATH_INFO: `/v3/projects/32567c92dfb44d1484b0aee7aacf0b47` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:36.609275 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32567c92dfb44d1484b0aee7aacf0b47)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:36.611508 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:36.665983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32567c92dfb44d1484b0aee7aacf0b47', 'request_id': 'req-8d2983f3-72ee-4131-96ab-888b7d54c606'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.673556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32567c92dfb44d1484b0aee7aacf0b47', 'request_id': 'req-8d2983f3-72ee-4131-96ab-888b7d54c606'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.679672 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32567c92dfb44d1484b0aee7aacf0b47', 'request_id': 'req-8d2983f3-72ee-4131-96ab-888b7d54c606'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.690967 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] The token cache is being invalidate because project 32567c92dfb44d1484b0aee7aacf0b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 11:53:36.691250 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.691899 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8d2983f3-72ee-4131-96ab-888b7d54c606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 11:53:36.694105 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1436/2866] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 11:53:36 2022] DELETE /identity/v3/projects/32567c92dfb44d1484b0aee7aacf0b47 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:53:36.699003 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b8b01-cae9-4f20-a47a-8774a06cb46e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:36.699290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b8b01-cae9-4f20-a47a-8774a06cb46e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:36.699562 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b8b01-cae9-4f20-a47a-8774a06cb46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:36.759361 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d61b8b01-cae9-4f20-a47a-8774a06cb46e None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:53:36.760505 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d61b8b01-cae9-4f20-a47a-8774a06cb46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:36.826139 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1431/2867] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:53:36.834181 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8284e-d0f0-4c85-bf4d-540bd174464b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:36.834476 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8284e-d0f0-4c85-bf4d-540bd174464b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:36.834758 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8284e-d0f0-4c85-bf4d-540bd174464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:36.853666 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-78b8284e-d0f0-4c85-bf4d-540bd174464b None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:53:36.854939 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-78b8284e-d0f0-4c85-bf4d-540bd174464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:36.863167 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1437/2868] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:53:36.868429 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b8b01-cae9-4f20-a47a-8774a06cb46e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:36.878346 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b8b01-cae9-4f20-a47a-8774a06cb46e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:36.883883 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:36.884581 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:36.884869 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:36.885149 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:36.885924 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 17 11:53:36.886245 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:36.887847 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:36.900745 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1432/2869] 10.222.0.25 () {58 vars in 1152 bytes} [Fri Jun 17 11:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 17 11:53:36.906190 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8284e-d0f0-4c85-bf4d-540bd174464b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:36.916536 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8284e-d0f0-4c85-bf4d-540bd174464b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:36.922135 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:36.922872 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:36.923299 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:36.923526 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:36.924214 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:36.926333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:36.937673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:36.955308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1438/2870] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:53:36 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:53:36.959628 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:36.969746 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce63d-c124-4332-a6e7-4f1fe74b1650 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:37.016462 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:37.035631 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:37.036865 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:37.037153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:37.037427 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:37.038168 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:37.040114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:37.060594 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1433/2871] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:53:36 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 17 11:53:37.064701 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:37.075085 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1bc68-1dd9-4a47-b3c8-a686fc2d1e45 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:37.079993 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:37.080662 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:37.080944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:37.081218 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:37.082005 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:37.083594 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:37.094296 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1439/2872] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:53:37.097975 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:37.110769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24639a55-ba81-4b97-afed-65290676dfaa admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:37.116029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c02f3-fc99-4ce7-bdef-3d32715a2f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:37.116698 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c02f3-fc99-4ce7-bdef-3d32715a2f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:37.116984 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c02f3-fc99-4ce7-bdef-3d32715a2f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:37.117256 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c02f3-fc99-4ce7-bdef-3d32715a2f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:37.118101 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c02f3-fc99-4ce7-bdef-3d32715a2f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:37.119717 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c02f3-fc99-4ce7-bdef-3d32715a2f4e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:37.129967 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1434/2873] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:53:37.134143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:37.144308 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bae36b-f58e-4c75-9c09-b6a2e5222299 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:37.148926 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:37.149805 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:37.150123 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:37.150406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] PATH_INFO: `/v3/projects/62f3ea3d9adc43e999426fe7da9e8a82/users/f88be2ec58c54d3d84c5ac84329351cd/roles/9ebaaf52f4cd402880b403c50600168e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:37.165695 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f3ea3d9adc43e999426fe7da9e8a82, user_id=f88be2ec58c54d3d84c5ac84329351cd, role_id=9ebaaf52f4cd402880b403c50600168e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:37.168852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:37.184590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1440/2874] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:53:37 2022] PUT /identity/v3/projects/62f3ea3d9adc43e999426fe7da9e8a82/users/f88be2ec58c54d3d84c5ac84329351cd/roles/9ebaaf52f4cd402880b403c50600168e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:53:37.192339 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe2943-321c-4233-b53a-8126f7c55982 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:37.192630 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe2943-321c-4233-b53a-8126f7c55982 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:37.192911 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe2943-321c-4233-b53a-8126f7c55982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:37.211503 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ddbe2943-321c-4233-b53a-8126f7c55982 None None] MFA Rules not processed for user `4a953762447e42cfa82713049dc178a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:53:37.212703 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ddbe2943-321c-4233-b53a-8126f7c55982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:37.273114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1435/2875] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:53:37.282548 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:37.284495 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:37.346519 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c47ead-f309-4745-80b2-56a5e251e7f0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:37.350792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:37.351439 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:37.351724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:37.351998 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:37.360942 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:37.362406 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:37.371713 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1441/2876] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:41.416063 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7e55-1d1b-4a86-ad8b-746c7cd99601 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:41.416625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7e55-1d1b-4a86-ad8b-746c7cd99601 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:41.416625 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e7e55-1d1b-4a86-ad8b-746c7cd99601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:41.446927 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2b8e7e55-1d1b-4a86-ad8b-746c7cd99601 None None] MFA Rules not processed for user `f88be2ec58c54d3d84c5ac84329351cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:53:41.448178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2b8e7e55-1d1b-4a86-ad8b-746c7cd99601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:41.516838 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1436/2877] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:53:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:53:41.521964 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:41.532150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3e2fd-b986-44c5-86c1-ec8eab9521f7 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:41.537243 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:41.537944 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:41.538229 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:41.538506 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:41.539208 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:41.541112 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:41.572449 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1442/2878] 10.222.0.25 () {60 vars in 1152 bytes} [Fri Jun 17 11:53:41 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:53:41.573602 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e7e55-1d1b-4a86-ad8b-746c7cd99601 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:41.585614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e7e55-1d1b-4a86-ad8b-746c7cd99601 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:41.639522 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:41.653135 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:41.653866 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:41.654158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:41.654442 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:41.655158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:41.657129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:41.688688 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1437/2879] 10.222.0.25 () {60 vars in 1143 bytes} [Fri Jun 17 11:53:41 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 17 11:53:41.689611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:41.701439 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427ce06-5fc6-4707-bd64-72710d263368 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:41.706985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:41.707755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:41.708883 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:41.709140 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:41.709975 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:41.711611 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:41.725914 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1443/2880] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 17 11:53:41.727982 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:41.738508 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a2fe-4250-4750-9933-9ed8a390d623 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:41.743414 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:41.744331 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:41.744620 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:41.744902 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] PATH_INFO: `/v3/projects/bce957049c2f4b208a8655758a850a91/users/87dbc294ba274fc18c78e2140c9205c1/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:41.760601 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce957049c2f4b208a8655758a850a91, user_id=87dbc294ba274fc18c78e2140c9205c1, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:41.763309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:41.776543 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1438/2881] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:53:41 2022] PUT /identity/v3/projects/bce957049c2f4b208a8655758a850a91/users/87dbc294ba274fc18c78e2140c9205c1/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:53:41.780456 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:41.790956 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d69891-ea49-4b56-b913-55a042d393ec admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:41.850607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82cc3e-ed3f-4273-b668-ad14caa24313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:41.851287 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82cc3e-ed3f-4273-b668-ad14caa24313 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:41.851575 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82cc3e-ed3f-4273-b668-ad14caa24313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:41.851853 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82cc3e-ed3f-4273-b668-ad14caa24313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:41.852577 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82cc3e-ed3f-4273-b668-ad14caa24313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:41.854234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82cc3e-ed3f-4273-b668-ad14caa24313 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:41.864082 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1444/2882] 10.222.0.25 () {58 vars in 1121 bytes} [Fri Jun 17 11:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 17 11:53:41.868780 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:41.879252 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868db54-ea25-457b-a8a5-e68eb725ac89 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:41.886517 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:41.887445 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:41.887930 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:41.888129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] PATH_INFO: `/v3/projects/bce957049c2f4b208a8655758a850a91/users/87dbc294ba274fc18c78e2140c9205c1/roles/a0f5f42c562a49678c833a3a1d6e816b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:41.890350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce957049c2f4b208a8655758a850a91, user_id=87dbc294ba274fc18c78e2140c9205c1, role_id=a0f5f42c562a49678c833a3a1d6e816b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:41.892654 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:41.903255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1439/2883] 10.222.0.25 () {60 vars in 1482 bytes} [Fri Jun 17 11:53:41 2022] PUT /identity/v3/projects/bce957049c2f4b208a8655758a850a91/users/87dbc294ba274fc18c78e2140c9205c1/roles/a0f5f42c562a49678c833a3a1d6e816b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 11:53:46.615960 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1a2fc-be56-4cc4-967e-3f2e43b8acc1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:46.616483 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1a2fc-be56-4cc4-967e-3f2e43b8acc1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:46.617983 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1a2fc-be56-4cc4-967e-3f2e43b8acc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:46.696016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c6e1a2fc-be56-4cc4-967e-3f2e43b8acc1 None None] MFA Rules not processed for user `87dbc294ba274fc18c78e2140c9205c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:53:46.697461 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c6e1a2fc-be56-4cc4-967e-3f2e43b8acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:46.721136 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:46.831615 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1445/2884] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:53:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:53:46.846485 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:53:46.849119 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:53:46.954106 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd4d6f-503a-459c-ba84-2883c07f4a1c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:53:46.964769 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:53:46.965787 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:53:46.966313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:53:46.966763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:53:47.053266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:53:47.055282 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:53:47.069511 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1440/2885] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:53:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:53:56.827871 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=24 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:53:57.067061 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:54:04.486971 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd467da-2ebc-4185-b317-e4eca706cff1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:54:04.487485 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd467da-2ebc-4185-b317-e4eca706cff1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:54:04.487610 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd467da-2ebc-4185-b317-e4eca706cff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:54:04.504775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fcd467da-2ebc-4185-b317-e4eca706cff1 None None] MFA Rules not processed for user `f88be2ec58c54d3d84c5ac84329351cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:54:04.506017 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fcd467da-2ebc-4185-b317-e4eca706cff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:54:04.512876 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1446/2886] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:54:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:54:14.499590 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:54:16.857137 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:54:16.858744 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:54:16.917313 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ed75-e55f-4c26-9914-2272857ed86a service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:54:16.920969 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:54:16.921622 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:54:16.921937 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:54:16.922220 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:54:16.923978 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:54:16.937464 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:54:16.938914 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:54:16.950389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1441/2887] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:54:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:54:19.304477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd467da-2ebc-4185-b317-e4eca706cff1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:54:19.314155 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd467da-2ebc-4185-b317-e4eca706cff1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:54:19.317874 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:54:19.318513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:54:19.318792 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:54:19.319151 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:54:19.321001 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:54:19.334381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:54:19.336315 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:54:19.347077 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1447/2888] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:54:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:54:19.642266 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:54:19.651737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b4366-440c-4b84-9e07-4ff6ea1f66df service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:54:19.655453 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398a57-560a-469c-b3d3-a87cea9f4a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:54:19.656100 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398a57-560a-469c-b3d3-a87cea9f4a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:54:19.656389 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398a57-560a-469c-b3d3-a87cea9f4a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:54:19.656670 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398a57-560a-469c-b3d3-a87cea9f4a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:54:19.666153 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91398a57-560a-469c-b3d3-a87cea9f4a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:54:19.667573 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91398a57-560a-469c-b3d3-a87cea9f4a8b service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:54:19.677185 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1442/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:54:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:54:21.548584 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:54:21.564144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b91c24-2669-4566-82b0-65a8a85788b9 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:54:21.571098 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:54:21.572189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:54:21.572540 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:54:21.572889 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:54:21.576025 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:54:21.592925 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:54:21.594382 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:54:21.603453 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1448/2890] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:54:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:54:29.672820 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=32 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:54:31.599529 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=12 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:56:19.071238 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:56:19.071737 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:56:19.071848 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:56:19.089349 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] MFA Rules not processed for user `f88be2ec58c54d3d84c5ac84329351cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:56:19.090613 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:56:19.097578 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1443/2891] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:56:19.415770 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:56:19.424724 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da8df0-ddb8-40d0-8b70-a8d188f8a158 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:56:19.428363 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:56:19.429016 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:56:19.429300 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:56:19.429574 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:56:19.431630 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:56:19.442144 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:56:19.443581 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:56:19.452556 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1449/2892] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:56:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:56:19.935338 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:56:19.937158 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:56:20.031473 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632971bd-b9af-4600-956d-97b7b73e7ef1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:56:20.038082 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:56:20.039140 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:56:20.040176 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:56:20.040493 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:56:20.042720 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:56:20.059939 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:56:20.063781 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:56:20.078867 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1444/2893] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:56:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:56:29.448775 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:56:30.070006 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=35 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:56:32.080125 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:56:32.081579 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:56:32.099207 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e3453-cc26-4a79-8749-ce0fe8c478b7 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:56:32.104658 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:56:32.105318 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:56:32.105607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:56:32.105912 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:56:32.107791 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:56:32.121234 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:56:32.122852 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:56:32.135247 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1450/2894] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:56:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:56:35.971692 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:56:35.984164 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aada84-0f94-4d54-81b5-50f399aa1af0 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:56:35.990029 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c353f-0b1e-4e38-8d40-c4dddeca455c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '804a35678c91438eae36f61c1de8c579', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:56:35.990767 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c353f-0b1e-4e38-8d40-c4dddeca455c service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:56:35.991124 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c353f-0b1e-4e38-8d40-c4dddeca455c service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:56:35.991470 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c353f-0b1e-4e38-8d40-c4dddeca455c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:56:35.993803 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5d4c353f-0b1e-4e38-8d40-c4dddeca455c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:56:36.007138 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c353f-0b1e-4e38-8d40-c4dddeca455c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:56:36.009364 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c353f-0b1e-4e38-8d40-c4dddeca455c service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:56:36.020475 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1445/2895] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:56:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:56:42.128985 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:56:46.015854 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:57:55.113053 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:57:55.114491 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:55.128460 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2181b-9e61-4e55-b682-366d20b52ec2 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:57:55.133512 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:57:55.134189 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:57:55.134477 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:57:55.134755 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:57:55.136507 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:57:55.148949 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:57:55.150591 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:57:55.161003 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1451/2896] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:57:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:58:05.156531 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:58:22.891741 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc09e1c-ff2d-4bd7-84b3-06ee4efb527d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:58:22.892316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc09e1c-ff2d-4bd7-84b3-06ee4efb527d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:58:22.892316 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc09e1c-ff2d-4bd7-84b3-06ee4efb527d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:58:22.910183 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fdc09e1c-ff2d-4bd7-84b3-06ee4efb527d None None] MFA Rules not processed for user `f88be2ec58c54d3d84c5ac84329351cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 17 11:58:22.911403 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fdc09e1c-ff2d-4bd7-84b3-06ee4efb527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 11:58:22.919269 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1446/2897] 10.222.0.25 () {58 vars in 957 bytes} [Fri Jun 17 11:58:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 17 11:58:32.905371 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 11:59:48.867896 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 11:59:48.877188 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0b825-7a42-4593-8839-a5a5026c3f79 service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 11:59:48.880786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 11:59:48.881440 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 11:59:48.881712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 11:59:48.882036 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 11:59:48.889236 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 11:59:48.890707 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 11:59:48.898622 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1452/2898] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 11:59:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 11:59:58.895137 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:00:11.114308 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc09e1c-ff2d-4bd7-84b3-06ee4efb527d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:11.124500 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc09e1c-ff2d-4bd7-84b3-06ee4efb527d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:11.128014 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:11.128651 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:11.128931 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:11.129203 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:11.139523 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:11.141563 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:11.151763 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1447/2899] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 12:00:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:00:12.147028 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:12.156225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2feec5-fab0-4e98-81b9-cc2c8778f33b service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:12.159933 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:12.160582 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:12.160884 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:12.161141 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:12.173333 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:12.174760 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:12.186293 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1453/2900] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 12:00:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:00:13.379212 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:13.388754 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cc0af-c8fd-4ab4-8784-3ad8e1ee674f service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:13.393290 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:13.393954 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:13.394240 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:13.394512 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:13.404860 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:13.406416 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:13.418012 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1448/2901] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 12:00:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:00:22.181054 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:00:23.412614 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=12 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:00:24.308225 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:24.318238 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6e74f-366f-411a-b681-9c2e345edc1c service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:24.321802 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b35d6d2cd848f4841352fbb58f2053', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:24.322479 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:24.322786 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:24.323056 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:24.383193 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:24.384552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:24.394084 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1454/2902] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 12:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:00:27.498051 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:27.512145 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315eb06-bc4c-409f-a5ed-fca304e69c15 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:27.518589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:27.519724 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:27.520194 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:27.520846 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:27.522794 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:00:27.537027 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:27.538862 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:27.558034 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1449/2903] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 12:00:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:00:30.638860 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:30.652844 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792ac3-0a47-4ca9-962a-e91688f26288 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:30.660565 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:30.661632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:30.661632 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:30.662050 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:30.664429 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:00:30.691943 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:30.693673 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:30.722066 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1455/2904] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 12:00:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:00:30.722391 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:30.734589 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43d2a-ebd7-4b96-802e-165a8383e311 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:30.743243 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:30.744307 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:30.744597 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:30.744873 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] PATH_INFO: `/v3/users/f88be2ec58c54d3d84c5ac84329351cd` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:30.746178 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88be2ec58c54d3d84c5ac84329351cd)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:30.748075 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:30.800680 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88be2ec58c54d3d84c5ac84329351cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:30.808637 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88be2ec58c54d3d84c5ac84329351cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:30.818141 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88be2ec58c54d3d84c5ac84329351cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:30.818850 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88be2ec58c54d3d84c5ac84329351cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:30.831350 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88be2ec58c54d3d84c5ac84329351cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:30.837277 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1450/2905] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 12:00:30 2022] DELETE /identity/v3/users/f88be2ec58c54d3d84c5ac84329351cd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:00:31.011607 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:31.026381 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8ad54-8f4f-4110-bd6a-58fb0fd50a0e service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:31.086280 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:31.087150 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:31.087435 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:31.087712 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] PATH_INFO: `/v3/projects/62f3ea3d9adc43e999426fe7da9e8a82` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:31.088882 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62f3ea3d9adc43e999426fe7da9e8a82)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:31.090821 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:31.135317 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62f3ea3d9adc43e999426fe7da9e8a82', 'request_id': 'req-17d55f80-555d-40a3-8d21-8e294ad92545'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.142762 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62f3ea3d9adc43e999426fe7da9e8a82', 'request_id': 'req-17d55f80-555d-40a3-8d21-8e294ad92545'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.148392 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62f3ea3d9adc43e999426fe7da9e8a82', 'request_id': 'req-17d55f80-555d-40a3-8d21-8e294ad92545'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.160165 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] The token cache is being invalidate because project 62f3ea3d9adc43e999426fe7da9e8a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:00:31.160447 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.161081 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.162654 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1456/2906] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 12:00:31 2022] DELETE /identity/v3/projects/62f3ea3d9adc43e999426fe7da9e8a82 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:00:31.166849 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:31.168312 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:00:31.181114 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9eb15-7998-4390-8069-39ef9bfea177 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:31.236945 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:31.237816 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:31.238129 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:31.238407 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] PATH_INFO: `/v3/users/87dbc294ba274fc18c78e2140c9205c1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:31.239586 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] RBAC: Authorizing `identity:delete_user(user_id=87dbc294ba274fc18c78e2140c9205c1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:31.241489 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:31.289598 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87dbc294ba274fc18c78e2140c9205c1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.296825 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87dbc294ba274fc18c78e2140c9205c1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.302828 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:00:31.306278 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87dbc294ba274fc18c78e2140c9205c1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.306989 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87dbc294ba274fc18c78e2140c9205c1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.320807 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87dbc294ba274fc18c78e2140c9205c1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.327314 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1451/2907] 10.222.0.25 () {58 vars in 1223 bytes} [Fri Jun 17 12:00:31 2022] DELETE /identity/v3/users/87dbc294ba274fc18c78e2140c9205c1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:00:31.507498 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:00:31.510341 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:00:31.525026 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d55f80-555d-40a3-8d21-8e294ad92545 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:00:31.533552 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:00:31.585984 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a953762447e42cfa82713049dc178a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5460a823bf554ac5a99fd5d73c21c6d1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:00:31.586858 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:00:31.587145 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:00:31.587423 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] PATH_INFO: `/v3/projects/bce957049c2f4b208a8655758a850a91` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:00:31.588617 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce957049c2f4b208a8655758a850a91)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:00:31.590678 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:00:31.638143 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce957049c2f4b208a8655758a850a91', 'request_id': 'req-2a966a92-f861-484e-8d62-2045a1a42afd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.645513 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce957049c2f4b208a8655758a850a91', 'request_id': 'req-2a966a92-f861-484e-8d62-2045a1a42afd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.650913 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce957049c2f4b208a8655758a850a91', 'request_id': 'req-2a966a92-f861-484e-8d62-2045a1a42afd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.661966 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] The token cache is being invalidate because project bce957049c2f4b208a8655758a850a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 17 12:00:31.662254 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.662879 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2a966a92-f861-484e-8d62-2045a1a42afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 17 12:00:31.664709 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1457/2908] 10.222.0.25 () {58 vars in 1232 bytes} [Fri Jun 17 12:00:31 2022] DELETE /identity/v3/projects/bce957049c2f4b208a8655758a850a91 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 17 12:00:41.323255 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:00:41.658849 dv-cinder-837356-3-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 17 12:02:59.097309 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 17 12:02:59.098778 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:02:59.238382 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f8a3-195e-4d01-8e32-984db63f910b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 17 12:02:59.299042 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d8d96-8c7b-4cb6-9c0a-2632efa7aa12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8bdac27bde4a2785d4aaad8213baca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8fc033695b4a4a09a596920a2b36c873', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 17 12:02:59.299696 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d8d96-8c7b-4cb6-9c0a-2632efa7aa12 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 17 12:02:59.299992 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d8d96-8c7b-4cb6-9c0a-2632efa7aa12 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 17 12:02:59.300272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d8d96-8c7b-4cb6-9c0a-2632efa7aa12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 17 12:02:59.302079 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-435d8d96-8c7b-4cb6-9c0a-2632efa7aa12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 17 12:02:59.314272 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d8d96-8c7b-4cb6-9c0a-2632efa7aa12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 17 12:02:59.315947 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d8d96-8c7b-4cb6-9c0a-2632efa7aa12 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 17 12:02:59.326001 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1452/2909] 10.222.0.25 () {60 vars in 1365 bytes} [Fri Jun 17 12:02:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 17 12:03:09.321569 dv-cinder-837356-3-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}